USOO7363649B2

(12) United States Patent (10) Patent No.: US 7,363,649 B2 Marsh (45) Date of Patent: Apr. 22, 2008

(54) MEDIA CONTENT DESCRIPTIONS 6,614,987 B1 9/2003 Ismail et al. 6,651,253 B2 * 1 1/2003 Dudkiewicz et al...... T25,114 (75) Inventor: David J. Marsh, Sammamish, WA 6,704,491 B1 3/2004 Revis (US) 6,842,761 B2 1/2005 Diamond et al. 6,973,665 B2 12/2005 Dudkiewicz et al. 2001/004.1980 A1 11/2001 Howard et al. (73) Assignee: Microsoft Corporation, Redmond, WA 2002/0042923 A1 4/2002 Asmussen et al. (US) 2002/0104087 A1 8, 2002 Schaffer et al. 2002fO140719 A1 * 10, 2002 Amir et al...... 345,716 (*) Notice: Subject to any disclaimer, the term of this 2002/0147984 A1 10, 2002 Tomsen et al. patent is extended or adjusted under 35 2002/0152463 A1 10, 2002 Dudkiewicz U.S.C. 154(b) by 0 days. (Continued) (21) Appl. No.: 11/119,170 FOREIGN PATENT DOCUMENTS (22) Filed: Apr. 29, 2005 JP 2000 067060 3, 2000 (65) Prior Publication Data (Continued) US 2005/O192987 A1 Sep. 1, 2005 OTHER PUBLICATIONS O O Jasinschi, R.S. et al., “Automatic TV Program Genre Classification Related U.S. Application Data Based on Audio Patterns’, IEEE 2001, 6 pages, Philips Research, (62) Division of application No. 10/125,260, filed on Apr. Scarborough Road, Briarcliff Manor, NY. 10510, USA. 16, 2002, now Pat. No. 7,073,193. (Continued) (51) Int. weCl PrimaryFIFOF EXOFiliefExaminer CharlesaS RExOS G06F 7/30 (2006.01) Assistant Examiner Belix M Ortiz (52) U.S. Cl...... 725/114; 725/109; 707/104.1; (74) Attorney, Agent, or Firm—Lee & Hayes, PLLC 707/100 (58) Field of Classification Search ...... 725/114 (57) ABSTRACT 725/138, 144; 345/716 See application file for complete search history. A media content description system receives media content descriptions from one or more metadata providers. The (56) References Cited media content description system associates each media content description with the metadata provider that provided U.S. PATENT DOCUMENTS the description. The media content description system may 5,969,748 A 10, 1999 Casement et al. generate composite descriptions based on the received 5,973,683 A 10/1999 Cragun et al. media content descriptions. The media content description 5,995,133 A 11, 1999 Kim 6,115,057 A 9, 2000 Kwoh et al. system provides data associated with the media content 6,157,411 A 12/2000 Williams et al. descriptions to one or more program data providers. 6,317,885 B1 11/2001 Fries 6,457,010 B1 9/2002 Eldering et al. 20 Claims, 17 Drawing Sheets

Media Cortent 304 Content description description System Metadata Provider(s) Sci. 302 300 y o iders 306 EPG. Database EPG Sewer gontent 308 3.18 Providers Stored Content 322 Content Distribution System 310 Content Data Broadcast Content Server Processor(s) Transmitter 320 328 324

328 Broadcast Network

C 312(1) 312(2) 312(N) se o

332 330(1) 33O2) as s es s 334 ess US 7,363,649 B2 Page 2

U.S. PATENT DOCUMENTS Liu et al., “Classifying Video Documents. By Hierarchical Structure Of Video Contents'. The Computer Journal, vol.43, No. 5, 2000, 15 2003/OO93694 A1* 5/2003 Medvinsky et al...... T13 201 pageS. 2003/O135857 A1 7/2003 Pendahur et al. Truong et al., “Automatic Genre Identification for Content-Based 2005, 0028194 A1 2/2005 Elenbaas et all Video Categorization', IEEE, 2000, 4 pages. 2005/O192987 A1 9, 2005 Marsh De Jong et al., “Die technische Fachzeitschrift fur Fernsehen, Film 2006, O190966 A1 8, 2006 McKissicket al. und elektronische Medien'. Fernseh-Und Kino-Technik-55, Jahrgang, Nr. Aug. 9, 2001, 10 pages. FOREIGN PATENT DOCUMENTS Mulder, P. “The Integration Of Metadata from Production To JP 2001 134582 5, 2001 Consumer', EBU Technical Review, Sep. 2000, 5 pages. WO WOOOf 40026 T 2000 Evain, J., “TV-Anytime Metadata, A Preliminary Specification On WO WO O1/47257 6, 2001 Schedule”, EBU Technical Review, Sep. 2000, 14 pages, EBU WO WO O2,11445 2, 2002 Technical Department. Herla, Von Siegbert, "Online-Archive-MIT Metadaten Zum Erfolg', OTHER PUBLICATIONS Dec. 15, 2000, 8 pages. Joung, Ye Sun, “A Metadata Repository System For An Efficient Wei et al., “TV Program Classification Based on Face and Text Description Of Visual Multimedia Documents”. Concurrent Engi Processing, 2000, 4 pages; Computer Science Department, Wayne neering, Research and Applications, vol. 9, No. 2, Jun. 2001, 12 University, Detroit MI, Philips Research, Briarcliff Manor, NY. pageS. Taskiran et al., “Discovering Video Structure Using The Psuedo Adami et al., “The ToCAI Description Scheme For Indexing And Semantic Trace'. Proceedings of SPIE, vol. 4315, 2001, 8 pages. Retrieval Of Multimedia Documents', Multimedia Tools And McDonald et al., “Online Television Library: Organisation and Applications, 14, 153-173, 2001. Content Browsing for General Users'. Proceedings of SPIE, vol. 4315, 2001, 9 pages. * cited by examiner U.S. Patent Apr. 22, 2008 Sheet 1 of 17 US 7,363,649 B2

PROGRAM DATA

CONTENT DESCRIPTION METADATA 1

STORY LINE DIRECTOR CRITIC OPINIONS

REVIEWS

RECOMMENDATION S

INSTANCE DESCRIPTION METADATA 104

TIME

CHANNEL U.S. Patent Apr. 22, 2008 Sheet 2 of 17 US 7,363,649 B2

REVIEWS

204

CRITIC OPINIONS

RECOMMENDATIONS

204

STORY LINE

RATINGS

DIRECTOR

ACTORS U.S. Patent Apr. 22, 2008 Sheet 3 of 17 US 7,363,649 B2

Media Content 304 Content Description Description System Metadata Provider(s) 302

Program Data Provider(s) 306

EPG Database EPG Server Content 308 316 318 Provider(s)

Content Distribution System 310 Broadcast Content Server Processor(s) Transmitter 320 326 324 | 314 328

Broadcast NetWork

312(1) 312(2) 312(N) U.S. Patent Apr. 22, 2008 Sheet 4 of 17 US 7,363,649 B2

Media Content Description System

Metadata Provider Program Data Interface(s) Provider interface(s) Processor(s) O2 404

Operating Media Person Data Repository 10 414

Content Description Metadata Provider Metadata Repository Data Repository 412 416 U.S. Patent Apr. 22, 2008 Sheet 5 of 17 US 7,363,649 B2

Content Description Metadata Repository XML File Details MCID(1) Metadata Category 1 Category 1 Entity (Element 1, Element 2, ...) Category 1 Entity (Element 1, Element 2, ...) Metadata Category 2 Category 2 Entity (Element 1, Element 2, ...) Category 2 Entity (Element 1, Element 2, ...)

Metadata Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MCID(2) Metadata Category 1 Category 1 Entity (Element 1, Element 2, ...) Category 1 Entity (Element 1, Element 2, ...) Metadata Category 2 Category 2 Entity (Element 1, Element 2, ...) Category 2 Entity (Element 1, Element 2, ...) Metadata Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MCID(N) U.S. Patent Apr. 22, 2008 Sheet 6 of 17 US 7,363,649 B2

Associated People Genre Data Content Descriptions 604 606 608

102 \, Popularity 1O

Censor Ratings 12 ldentifying Data 602 Technical Details 614

Production Details 616

Schema Implementation Language Data Enhancements Details 618 U.S. Patent Apr. 22, 2008 Sheet 7 Of 17 US 7,363,649 B2

identifvind Data Genre Data 602 60 Alternate Content dentifiers Genre Program Type Title Genre Degrees Episode Name Genre intent Album Target Audience Version Detail Year Set Part Detail Era Set Location Portrayed From The Book Degree of True Story Degree of Animated 2D Degree of Animated 3D Degree of Puppet Characters Degree of International Event Associated People Degree of Sophistication Genre Music Genre Photo or Home Movie

Content Descriptions 608 Popularity Plot One Line Abstract 1 O Plot Long Abstract Keywords Critic Reviews Content Note Popularity Availability Date Broadcast Ubiquity Content Duration U.S. Patent Apr. 22, 2008 Sheet 8 of 17 US 7,363,649 B2

Censor Ratings 12 Censor Parental Rating Extended Censor Rating Adult Content Extended Censor Rating Bad Language Extended Censor Rating Violence Extended Censor Rating Nudity Extended Censor Rating Drug Abuse

Technical Details 14 Capture Mechanism Image Capture Resolution Video Capture Temporal Rate Hz Video Captured Using Interlace Sound Capture Sampling Sound Capture Compression Camera Used Image Capture Compression Recorded Live Black and White Silent Post Production Processing Special Electronic Processing Aspect Ratio Aspect Ratio Implementation Pan and Scan Vectors Origination Note Stereo Sound Surround Sound U.S. Patent Apr. 22, 2008 Sheet 9 Of 17 US 7,363,649 B2

Production Details Language Data

Made For Language Primary Original Budget Language Segment Original Box Office Language Dub Production Company Language Audio Track Distribution Company Language Text Burnt in Principal Country of Main Production Company language Text Track Capture Location Language Description Track Copyright Sign Language Track URL Production Company

Enhancements 18 ATVEF Data Enhancement Educational Commands Educational Informational Enhancements Schema Implementation Multiple Camera Angles Details 622 Multiple Story Lines Phone in XPath Content ArtWork URL Enhancements XPath Video Trailer URL More info XPath Chapter Thumbnails ASSOciated Phone Number XPath Time interval Thumbnails Associated Teletext Page Number XPath Script Or Lyrics VCR Programming Code XPath Original Storyboard U.S. Patent Apr. 22, 2008 Sheet 10 of 17 US 7,363,649 B2

Media Person Data Repository

XML. Person File Details MPI (1) Person Category 1 Category 1 Entity (Element 1, Element 2, ... Category 1 Entity (Element 1, Element 2, ... Person Category 2 Category 2 Entity (Element 1, Element 2, ... Category 2 Entity (Element 1, Element 2, ...

Person Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MPI (2) Person Category 1 Category 1 Entity (Element 1, Element 2, ...) Category 1 Entity (Element 1, Element 2, ...) Person Category 2 Category 2 Entity (Element 1, Element 2, ...) Category 2 Entity (Element 1, Element 2, ...)

Person Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MPI (N) U.S. Patent Apr. 22, 2008 Sheet 11 of 17 US 7,363,649 B2

Person Categories 1900

Name Gender Marita Status Ethnic Origin Religion Height Birth Date Birth Place Alive Death Date Death Cause Citizenship Residence Place Related PerSon Biography One Line Biography long Official Home Page URL Fan Site URL More information URL Email Address Office PhoneNumber Home Phone Number Fax Number XPath Person ArtWork XPath Person Video U.S. Patent Apr. 22, 2008 Sheet 12 of 17 US 7,363,649 B2

Metadata Provider Data Repository 416

MSI (1) Identifying Data Metadata Category Trust Level 1 Metadata Category Trust Level 2 Metadata Category Trust Level (N) MSI (2) identifying Data Metadata Category Trust Level 1 Metadata Category Trust Level 2 Metadata Category Trust Level (N) MSI (N) ldentifying Data Metadata Category Trust Level 1 Metadata Category Trust Level 2 Metadata Category Trust Level (N) U.S. Patent Apr. 22, 2008 Sheet 13 of 17 US 7,363,649 B2

Metadata Category Trust Levels

Title Episode Version Album Parts Artists Director Producer Editor Script Writer Lyrics Writer ldentifying Data 2100 Music Composer Location Company Name Date Company URL Duration Scheme Details URL Format Email Contact 1 Genre Degrees Email Contact 2 Genre Program Type Email Contact 3 Genre intent Address Line 1 Genre Target Audience Address Line 2 Genre Attributes Address Town Review Address State Popularity Adderss Zip Code Broadcast Ubiquity Censor Parental Ratings Extended Censor Ratings Origination Features Copyright Textual Description Links Made For Budget Box Office Production Company Distribution Company Language Paths U.S. Patent Apr. 22, 2008 Sheet 14 of 17 US 7,363,649 B2 23O2 ? 2300 Receive metadata from

metadata provider

Does metadata NO include person description data?

Identify data associated with a person.

Populate media person data repository

Additional people described in received metadata?

Populate content description metadata repository U.S. Patent Apr. 22, 2008 Sheet 15 Of 17 US 7,363,649 B2

Populate media person data repository ? 2308

Determine an MP associated 2402 with the person

ldentify a category for which 2404 metadata was received

Create an entity for the person 2406 Category in the data repository

2408 2410

Did metadata Y already exist ina this person eS Generate Composite Entity Category?

2412

ls there data aSSOciated With another Yes person category?

No U.S. Patent Apr. 22, 2008 Sheet 16 of 17 US 7,363,649 B2

Populate content description metadata repository -2312

25O2 Generate an MCID associated With the media Content

ldentify a category for which 2504 data was received

2506 Create an entity for the metadata category

2508 2410

Did an entity already exist in this Generate Composite Entity metadata Category2

ls there data associated with another Yes metadata Category? U.S. Patent Apr. 22, 2008 Sheet 17 Of 17 US 7,363,649 B2

Generate Composite Entity

ldentify an element of an 26O2 entity in the category

2604 Determine the distinct Values for the element

2608

is there more than One Set the composite element distinct value? value to the One distinct value

Yes

2610 For each distinct value determine the sum of the provider trust levels

2612 Set the composite element value to the value with the highest trust level summation

Are there more elments associated with the entity? US 7,363,649 B2 1. 2 MEDIA CONTENT DESCRIPTIONS FIG. 8 illustrates an example associated people category. FIG. 9 illustrates example genre data categories. CROSS REFERENCE TO RELATED FIG. 10 illustrates example content descriptions catego APPLICATIONS ries. FIG. 11 illustrates example popularity categories. This application is a divisional of, and claims priority to, FIG. 12 illustrates example censor ratings categories. U.S. patent application Ser. No. 10/125,260, entitled “Media FIG. 13 illustrates example technical details categories. Content Descriptions' to Marsh, filed Apr. 16, 2002 now FIG. 14 illustrates example production details categories. U.S. Pat. No. 7,073,193, the disclosure of which is incor FIG. 15 illustrates example enhancements categories. porated by reference herein. 10 FIG. 16 illustrates example language data categories. FIG. 17 illustrates example schema implementation TECHNICAL FIELD details categories. FIG. 18 illustrates the structure of media person data This invention relates to media entertainment systems stored in an exemplary media person data repository. and, in particular, to architectures that Support media content 15 description metadata. FIG. 19 illustrates exemplary media person data catego 1S. BACKGROUND FIG. 20 illustrates the structure of metadata provider data stored in an exemplary metadata provider data repository. Many media entertainment systems provide electronic FIG. 21 illustrates examples of identifying data that may programming guides (EPGs) that allow users to interactively be stored in association with a metadata provider. select programs that they are interested in. Systems that FIG. 22 illustrates examples of metadata category trust employ EPG technology typically display programs orga levels that may be stored in association with a metadata nized according to the channel on which the program will be provider. broadcast and the time at which the broadcast will occur. 25 FIG. 23 is an example flow diagram that illustrates a Information identifying a particular program typically process for storing media content description metadata. includes the program title, and possibly a short description FIG. 24 is an example flow diagram that illustrates a of the program. process for storing metadata that describes a person associ Over time, a large amount of descriptive data may be ated with media content. generated that is associated with a particular piece of media 30 FIG. 25 is an example flow diagram that illustrates a content (e.g., a movie). This data may include, for example, process for storing metadata that describes the media con reviews of the content by various critics, user recommen tent. dations, rating information, and genre classifications. A user FIG. 26 is an example flow diagram that illustrates a may desire to make decisions about which programs to view process for generating a composite metadata entity. based on descriptive data associated with programs beyond 35 the short descriptions typically displayed in an EPG. DETAILED DESCRIPTION

SUMMARY Introduction The following describes a media content description An architecture that Supports media content descriptions 40 system. The media content description system stores meta is described. The media content description architecture has data related to media content (e.g., a movie, a television a system to receive metadata that describes media content show, or a song). The metadata that the media content from one or more metadata providers. The system typically description system stores is related to the content of the generates composite descriptions based on received meta media, independent of when or where the media content is data. The system provides media content descriptions to one 45 available. For example, the metadata stored in the media or more program data providers. content description system may include the name of a movie, a list of actors in the movie, the name of the director BRIEF DESCRIPTION OF THE DRAWINGS of the movie, and one or more critic reviews of the movie. The metadata stored in the media content description system The same numbers are used throughout the drawings to 50 does not include television channels or times when a par reference like features and components. ticular movie will be broadcast. The media content descrip FIG. 1 illustrates two categories of program data that can tion system provides the stored metadata to content distri be associated with media. bution systems, which provide the metadata, as well as the FIG. 2 is an example diagram illustrating the collection of associated media content to users. content description metadata associated with a piece of 55 Content Description Metadata Collection media over time. FIG. 1 illustrates two categories of program data 100 that FIG. 3 is a block diagram of an exemplary media enter can be associated with media. Media (e.g., a movie) can be tainment system that facilitates distribution of media and associated with content description metadata 102 and metadata associated with the media to multiple users. instance description metadata 104. Content description FIG. 4 is a block diagram of selected components of the 60 metadata 102 is data pertaining to the media content, for media content description system shown in FIG. 3. example, the director, actors, story line, ratings, critic opin FIG. 5 illustrates the structure of media content descrip ions, reviews, and recommendations pertaining to a movie. tion metadata stored in an exemplary content description Instance description metadata 104 is data pertaining to when metadata repository. and where the media content is available. For example, the FIG. 6 illustrates related types of content description 65 time and television channel on which a particular movie will metadata categories. be broadcast. Because content description metadata 102 is FIG. 7 illustrates example identifying data categories. associated with the content of media, and not a particular US 7,363,649 B2 3 4 broadcast of the media, content description metadata may be or more metadata providers 302. In one implementation, the maintained and updated throughout the life of a particular media content description system 304 generates composite media content. metadata based on metadata received from a plurality of FIG. 2 illustrates the collection of content description metadata providers 302. metadata associated with media over time. Content descrip Media content description system 304 provides the media tion metadata is first available when the media 202 is content description metadata to program data provider 306. created. For example, the name of a movie and the actors in In one implementation, the media content description sys the movie is content description metadata that is available tem 304 provides to program data provider 306, all of the when a movie is produced. The content description metadata stored metadata that is associated with a piece of media can be stored in a repository 204 over the life of the media. 10 content (e.g., a movie). For example, when a movie is produced, the name of the In another implementation, the media content description director, the names of the actors, a description of the story system 304 provides only the stored metadata that is asso line, and the rating of the movie may be stored in the ciated with the media content that was received from a repository 204 as content description metadata. Over time, particular metadata provider. more content description metadata may become available, 15 In an alternate implementation, each metadata provider is and can be added to the repository 204. For example, after assigned one or more trust levels or rankings, which may be the movie is released, critic opinions and recommendations associated with particular metadata categories that indicate may become available. Because this is information related to how trustworthy metadata received from that metadata pro the media content itself (and not just a particular broadcast vider is. In this implementation, the metadata that is pro or showing of the media content), this information is added vided to the program data provider may be based on the trust to the repository 204. At a still later point in time, additional levels associated with the metadata providers from which reviews of the media content may become available. These the metadata was received. reviews may also be added to the repository 204. In yet another implementation, media content description Content description metadata is generated by many dif system 304 generates composite metadata based on meta ferent Sources (e.g., movie production companies, movie 25 data received from a plurality of metadata providers, and critics, television production companies, individual viewers, provides the composite metadata to the program data pro etc.). A media content description system stores content vider. description metadata from multiple sources, and makes the Various implementations are contemplated, in which, the content description metadata available to users. The follow media content description system 304 may provide any ing discussion is directed to media-based entertainment 30 combination of content description metadata to program systems, such as interactive TV networks, cable and satellite data provider 306. networks that utilize electronic program guides, and Web Program data provider 306 includes an electronic program enabled TV networks. Client devices in such systems range guide (EPG) database 316 and an EPG server 318. The EPG from full-resource clients with substantial memory and database 316 stores electronic files of program data, which processing resources, such as TV-enabled personal comput 35 is used to generate an electronic program guide (or, "pro ers and personal video recorders (PVRs) equipped with gram guide'). The program data stored by the EPG database hard-disks, to low-resource clients with limited memory will be referred to as EPG data and may include content and/or processing resources. Such as traditional set-top description metadata 102 and instance description metadata boxes. While aspects of the described systems and proce 104. For example, the EPG database 316 may store program dures can be used in any of these systems and for any types 40 titles, ratings, characters, descriptions, actor names, station of client devices, they are described in the context of the identifiers, channel identifiers, schedule information, and so following exemplary environment. O. Exemplary Environment The EPG server 318 processes the EPG data prior to FIG. 3 illustrates an exemplary environment 300 in which distribution to generate a published version of the EPG data the methods, systems, and data structures described herein 45 which contains programming information for all channels may be implemented. Exemplary environment 300 is a for one or more days. The processing may involve any media entertainment system that facilitates distribution of number of techniques to reduce, modify, or enhance the EPG media and metadata associated with the media to multiple data. Such processes might include selection of content, users. The environment 300 includes one or more content content compression, format modification, and the like. The description metadata providers 302, a media content 50 EPG server 318 controls distribution of the published ver description system 304, one or more program data providers sion of the EPG data from program data provider 306 to the 306, one or more content providers 308, a content distribu content distribution system 310 using, for example, a file tion system 310, and multiple client devices 312(1), transfer protocol (FTP) over a TCP/IP network (e.g., Inter 312(2), . . . , 312(N) coupled to the content distribution net, UNIX, etc.). system 310 via a broadcast network 314. 55 Content provider 308 includes a content server 320 and Content description metadata provider 302 provides con stored content 322. Such as movies, television programs, tent description metadata associated with media to media commercials, music, and similar media content. Content content description system 304. Example content descrip server 320 controls distribution of the stored content 322 tion metadata providers are movie production companies, from content provider 308 to the content distribution system movie distribution companies, movie critics, and music 60 310. Additionally, content server 320 controls distribution of production companies. Any person, company, system, or live media content (e.g., content that was not previously entity that is able to generate or Supply media content stored, Such as live feeds) and/or media content stored at description metadata may be considered a content descrip other locations to the content distribution system 310. tion metadata provider 302. Content distribution system 310 contains a broadcast Media content description system 304 stores media con 65 transmitter 324 and one or more content and program data tent description metadata associated with a plurality of processors 326. Broadcast transmitter 324 broadcasts sig metadata categories and stores metadata received from one nals, such as cable television signals, across broadcast US 7,363,649 B2 5 6 network 314. Broadcast network 314 can include a cable viewer. With an EPG application, the TV viewer can look at television network, RF, microwave, satellite, and/or data schedules of current and future programming, set reminders network, Such as the Internet, and may also include wired or for upcoming programs, and/or enter instructions to record wireless media using any broadcast format or broadcast one or more television shows. protocol. Additionally, broadcast network 314 can be any Exemplary Media Content Description System type of network, using any type of network topology and any FIG. 4 illustrates selected components of media content network communication protocol, and can be represented or description system 304 shown in FIG. 3. Media content otherwise implemented as a combination of two or more description system 304 includes one or more metadata networks. provider interfaces 402 that facilitate communication Content and program data processor 326 processes the 10 between media content description system 304 and one or media content and EPG data received from content provider more metadata providers 302. Media content description 308 and program data provider 306 prior to transmitting the system 304 also includes one or more program data provider media content and EPG data across broadcast network 314. interfaces 404 that facilitate communication between media A particular content processor may encode, or otherwise content description system 304 and one or more program process, the received content into a format that is understood 15 data providers 306. by the multiple client devices 312(1), 312(2), . . . , 312(N) Media content description system 304 includes one or coupled to broadcast network 314. Although FIG. 3 shows more processors 406 and one or more memory components a single program data provider 306, a single content provider 408. Examples of possible memory components include a 308, and a single content distribution system 310, environ random access memory (RAM), a disk drive, a mass storage ment 300 can include any number of program data providers component, and a non-volatile memory (e.g., ROM. Flash, and content providers coupled to any number of content EPROM, EEPROM, etc.). Alternative implementations of distribution systems. the media content description system can include a range of Content distribution system 310 is representative of a processing and memory capabilities, and may include more head end service that provides EPG data, as well as media or fewer types of memory components than those described. content, to multiple subscribers. Each content distribution 25 Processor(s) 406 process various instructions to control the system 310 may receive a slightly different version of the operation of the media content description system 304 and EPG data that takes into account different programming to communicate with other electronic and computing preferences and lineups. The EPG server 318 creates differ devices. ent versions of EPG data (e.g., different versions of a An operating system 410, a content description metadata program guide) that include those channels of relevance to 30 repository 412, a media person data repository 414, and a respective head end services. Content distribution system metadata provider data repository 416 may be stored in 310 transmits the EPG data to the multiple client devices memory 408 and executed on processor 406. Content 312(1), 312(2). . . . , 312(N). In one implementation, for description metadata repository 412 stores structured con example, distribution system 310 utilizes a carousel file tent description metadata associated with media content. system to repeatedly broadcast the EPG data over an out 35 Media person data repository 414 stores structured data of-band channel to the client devices 312. identifying people who are associated with media content Client devices 312 can be implemented in multiple ways. (e.g., actors, directors, etc.). Metadata provider data reposi For example, a client device 312(1) receives broadcast tory 416 stores structured data that describes relationships content from a satellite-based transmitter via a satellite dish between content description metadata providers 302. In 328. Client device 312(1) is also referred to as a set-top box 40 alternate implementations, data repositories 412, 414, and or a satellite receiving device. Client device 312(1) is 416 may be implemented as one or more data repositories. coupled to a television 330(1) for presenting the content Exemplary Content Description Metadata Repository received by the client device, such as audio data and video Content description metadata repository 412 stores meta data, as well as a graphical user interface. A particular client data associated with the content of media. The data reposi device 312 can be coupled to any number of televisions 330 45 tory may be implemented as a relational database, an object and/or similar devices that can be implemented to display or oriented database, a set of one or more data files, one or more otherwise render content. Similarly, any number of client XML files based on an XML schema, or any other data devices 312 can be coupled to a television 330. structure method. For the purposes of this discussion, an Client device 312(2) is also coupled to receive broadcast exemplary content description metadata repository will be content from broadcast network 314 and communicate the 50 described as an XML file. received content to associated television 330(2). Client FIG. 5 illustrates the structure of media content descrip device 312(N) is an example of a combination television332 tion metadata stored in an exemplary content description and integrated set-top box 334. In this example, the various metadata repository. components and functionality of the set-top box are incor XML File Details porated into the television, rather than using two separate 55 The XML File Details metadata entity is used to store data devices. The set-top box incorporated into the television associated with the XML file in which the content descrip may receive broadcast signals via a satellite dish (similar to tion metadata is stored. An example XML File Details entity satellite dish 328) and/or via broadcast network 314. A personal computer may also be a client device 312 capable has the following elements: of receiving and rendering EPG data and/or media content. 60 In alternate implementations, client devices 312 may receive broadcast signals via the Internet or any other broadcast Content Description File Version Content Description Creator medium. Organization Date Time Content Description Created Language Used For Content Each client 312 runs an electronic program guide (EPG) Description application that utilizes the EPG data. An EPG application 65 Content Description Creator Person Schema Version Used enables a TV viewer to navigate through an onscreen program guide and locate television shows of interest to the US 7,363,649 B2 7 8 The Content Description File Version element stores a number that indicates the version of the file. As data is added TABLE 1-continued to a media content description over time, multiple versions ik Inupiak of the file may be stored. is Icelandic The Date Time Content Description Created element it Italian stores the date and time that the file was created. iu Inuktitut ja Japanese The Content Description Creator Person element stores iw Javanese the name of the person that created the file. ka. Georgian The Content Description Creator Organization element kk Kazakh stores the name of an organization that created the file. 10 kil Greenlandic km Cambodian The Language Used For Content Description element kn Kannada stores a value that indicates the language in which the ko Korean content description data is provided. In an exemplary sys kS Kashmiri tem, the value that is stored in the Language Used For ku Kurdish 15 ky Kirghiz Content Description element is a combination of a language a Latin code and name according to ISO 639. Examples include in Lingala "de German”, “es Spanish, and “en English'. An o Laothian t Lithuanian example list of language field values is given in Table 1, V Latvian Lettish below. mg Malagasy The Schema Version Used element stores a number that mi Maori indicates the version of an XML Schema associated with the mk Macedonian ml. Malayalam XML file. mn Mongolian mo Moldavian TABLE 1. mir Marathi 25 ms Malay Unknown mt Maltese None my Burmese aa Afar na Nauru ab Abkhazian ne. Nepali af Afrikaans in Dutch am Amharic 30 no Norwegian air Arabic oc. Occitan as Assamese ay Aymara az Azerbaijani ba. Bashkir pl Polish be Byelorussian 35 ps PashtoPushto bg Bulgarian pt Portuguese bh Bihari qu Quechua bi Bislama rm RhaetoRomance bn BengaliBangla rn Kirundi bo Tibetan ro Romanian br Breton ru Russian ca. Catalan 40 rw Kinyarwanda co Corsican Sa Sanskrit cs Czech sd. Sindhi cy Welsh Sg Sangho da Danish sh SerboCroatian de German Si Sinhalese dz Bhutani 45 sk Slovak el Greek sl Slovenian en English Sm Samoan eo Esperanto Sn Shona es. Spanish So Somali et Estonian sq Albanian eu Basque 50 Sr Serbian fa. Persian SS Siswati fi Finnish st Sesotho f Fiji Su Sundanese fo Faroese Sw Swedish fr French Sw Swahili fy Frisian 55 ta. Tamil ga Irish te. Telugu gd ScotsGaelic tg Tajik gl Galician th Thai gn Guarani ti Tigrinya gu Gujarati tk Turkmen ha Hausa tl Tagalog he Hebrew 60 tn Setswana hi Hindi to Tonga hr Croatian tr Turkish hu Hungarian tS Tsonga hy. Armenian tt Tatar ia Interlingua tw Twi id Indonesian 65 ug Uighur ie Interlingue uk Ukrainian US 7,363,649 B2 9 10 Media content description system 304 may receive con TABLE 1-continued tent description metadata, in the same metadata category, ur Urdu associated with the same media content, from a plurality of uz Uzbek metadata providers 302. These multiple values may repre vi Vietnamese sent different opinions as to the value of an attribute asso vo Volapuk ciated with the meida content. For example, two metadata wo. Wolof providers 302 may each have different titles associated with xh Xhosa yi Yiddish the same media content. For instance, for the same media yo Yoruba content, a movie, one metadata provider may associated the Za Zhuang 10 title, “Indiana Jones and the Temple of Doom' while another zh Chinese metadata provider may associate the title, “Indiana Jones: Zu Zulu the Temple of Doom' with the same media content. To Other Support multiple data values associated with each metadata category (e.g., multiple opinions as to the value of an MCID 15 attribute), the content description metadata repository 412 Each piece of media content is assigned a unique media Supports multiple entities within each metadata category. content identifier (MCID), such as MCID(1), Each entity includes one or more associated elements. MCID(2), . . . . and MCID(N). Based on the structure of In one implementation, the media content description content description metadata repository 412, the MCID system 304 determines the order of multiple entities within metadata entity stores one instance of one element, which is a metadata category based on category-specific rankings that the MCID. For example a particular movie would be iden are associated with content description metadata providers tified by a unique MCID. 3O2. The MCID may be implemented as any type of unique In an alternate implementation, the media content descrip identifier. In one implementation, the MCID is a concatena tion system 304 generates a composite entity based on tion of hexadecimal representations of specific metadata 25 entities received from a plurality of metadata providers 302. category values stored in the content description metadata When the media content description system 304 generates a repository. The metadata categories that are used to generate composite entity, it is listed first among a plurality of the MCID are assigned levels of significance within the entities, indicating that it has the highest rank. MCID. An exemplary MCID is based on data stored in the In one implementation, the composite is generated based Title, Episode Name, Version Detail, and Part Detail meta 30 on trust levels associated with the metadata providers 302 data categories, which are described with reference to FIG. that provided the metadata. Other ways of indicating relative 7. The MCID is a structured string of the form MCID TI order of entities within a metadata category may be used, TLE EPISODE VERSION PART, where TITLE, EPI including storing a rank indicator as part of the entity. SODE, VERSION, and PART are hexadecimal values gen Exemplary Content Description Metadata Categories erated based on values stored in the Title, Episode Name, 35 FIG. 6 illustrates related types of content description Version Detail, and Part Detail metadata category entities, metadata categories. Content description metadata 102 can respectively. include many types of related data, including identifying data 602, associated people 604, genre data 606, content An example MCID is: MCID 0050-004-c34d-47ef descriptions 608, popularity 610, censor ratings 612, tech 00d4 002f-83a5 03. In this example, “0050-004-c34d 40 nical details 614, production details 616, enhancements 618, 47ef is the TITLE portion of the MCID, based on data language data 620, and schema implementation details 622. stored in the value element of the Title metadata entity; Identifying data 602 includes any data that identifies media “00d4” is the EPISODE portion of the MCID, based on data content, such as a movie title. Other types of metadata (e.g., stored in the value element of the Episode Name metadata 604-622) associated with the media content are related to the entity: “002f-83a5” is the VERSION portion of the MCID, 45 based on data stored in the version reason element of the identifying data 602, and describe attributes of the media Version Detail metadata entity; and “03” is the PART portion COntent. of the MCID, based on data stored in the part element of the Identifying Data 602 Part Detail metadata entity. The TITLE portion of the MCID FIG. 7 illustrates example identifying data categories. (the 0050-004-c34d-47ef in the above example) is the most Identifying data 602 includes metadata categories that iden significant, followed by the EPISODE portion (the 00d4 part 50 tify the media content. Example identifying data categories in the above example), the VERSION portion (the 002f include alternate content identifiers, title, episode name, 83a5 in the above example), and finally, the PART portion album, version detail, and part detail. (the 03 in the above example), which is the least significant. Alternate Content Identifiers Related media can be determined by comparing portions of The Alternate Content Identifiers metadata category is the respective MCIDs. For example, for a television series, 55 used to map content identifiers assigned by other metadata all episodes of the series are assigned MCIDs that have the repositories to the MCID assigned by the media content same value in the TITLE portion, but different values in the description system 304. An example Alternate Content Iden EPISODE, VERSION, and PART portions. tifiers entity has the following elements: Metadata Category 1, Metadata Category 2..., Metadata 60 Category (N) Media content description metadata stored in content Value MSI description metadata repository 412 is structured according Entry Index Date Time to categories of data that may be associated with media Entry Tag content. These categories are represented in FIG. 5 as 65 Metadata Category 1, Metadata Category 2, . . . . Metadata The Value element stores a content identifier assigned to Category (N). the media content by another metadata repository. For US 7,363,649 B2 11 12 example, a metadata provider 302 may assign unique iden categories described include a Language element. Unless tifiers to media content within a proprietary data repository. stated otherwise, this description also applies to the Lan The Entry Index and Entry Tag elements are elements in guage element associated with metadata categories most metadata category entities and Support multiple entity described below. entries from one metadata provider, or in the case of a Episode Name composite description, the Entry Index and Entry Tag ele The Episode Name metadata category is used to store data ments may be used to differentiate between multiple opin identifying an episode name associated with the media ions as to the value of an attribute of the meida content. For content. An example entity within an Episode Name meta example, one metadata provider may assign multiple iden data category has the following elements: tifiers to the same media content. The metadata provider 10 specifies the order of importance of the multiple entries using the Entry Index field. The metadata provider can Value MSI provide information as to why each different entry exists in Entry Index Date Time the Entry Tag elements of each Alternate Content Identifiers Entry Tag Language entity. Alternatively, in a composite description, alternate 15 content identifiers received from multiple metadata provid ers may be distinguished based on the Entry Index and Entry The Value element stores a name or description of the Tag elements. episode. For example, for metadata describing an episode of The MSI element stores a metadata source identifier, the television series, “Friends, the Value element may which is a unique identifier assigned by the media content contain, “The one where Ross gets married, or for metadata description system 304 to identify each metadata provider describing an episode of the television series, “Frasier, the 302. The MSI, along with other data related to a metadata value element stores the actual name of the episode, for provider 302 is stored in metadata provider data repository example, “Bully for Martin.” 416. The structure of metadata provider data repository 416 The Entry Index and Entry Tag elements are described is described with reference to FIGS. 20-22. 25 above and indicate that a metadata provider may submit The Date Time element stores a date/time stamp indicat multiple opinions as to the episode name or description ing when the metadata associated with the Alternate Content associated with the media content, or a composite descrip Identifiers entity was received from the metadata provider tion may indicate various opinions as to the episode name, 3O2. the opinions submitted by multiple metadata providers. The above descriptions for the Entry Index, Entry Tag, 30 The MSI element is described above and identifies the MSI, and Date Time elements apply for each record metadata provider. described below that may contain any combination of these The Date Time element is described above and indicates fields. the date and time at which the data was received. Title The Language element is described above and identifies The Title metadata category is used to store one or more 35 the language in which the episode name or description is titles associated with the media content. An example entity given. within the Title metadata category has the following ele Album ments: The Album metadata category is used to store the title of a music album. This metadata category is used when storing 40 metadata associated with a music track, for example, a song. An example entity within the Album metadata category has Value MSI Entry Index Date Time the following elements: Entry Tag Language

45 The Value element stores a title associated with the media Value MSI Entry Index Date Time content. For example, the value element may contain, Entry Tag Language “Friends, to indicate the television series by that name, or may contain, “Indiana Jones and the Temple of Doom,” to indicate the movie by that name. 50 The Value element stores the title of the album associated The Entry Index and Entry Tag elements are described with the media content. above and indicate that a metadata provider may submit The Entry Index and Entry Tag elements are described multiple opinions as to the title associated with the media above and indicate that a metadata provider may submit content, or a composite description may indicate various multiple album titles associated with the media content. For opinions as to the title, the opinions submitted by multiple 55 example a recording artist may produce a song for a par metadata providers. For example, titles may be submitted in ticular album, and then at a later time, include the same song more than one language, or with different spellings for one on a greatest-hits album. In this case, the same metadata or more words in the title. provider may Submit metadata about the song, and relate the The MSI element is described above and identifies the Song to both the original album and the greatest-hits album metadata provider. 60 by Submitting both album titles. In a composite description, The Date Time element is described above and indicates the Entry Index and Entry Tag elements may differentiate the date and time at which the data was received. between various opinions as to the title, the opinions Sub The Language element stores an identifier of the language mitted by multiple metadata providers associated with the title stored in the Value element. As The MSI element is described above and identifies the described above, a standard set of language identifiers may 65 metadata provider. be used. An example of Such a standard set of language The Date Time element is described above and indicates identifiers is shown in Table 1. Many of the metadata the date and time at which the data was received. US 7,363,649 B2 13 14 The Language element is described above and identifies The Part element stores an integer indicating a part of the the language associated with the album title. media content. For example, for a movie aired in two parts Version Detail over two nights, the part aired on the first night is identified The Version Detail metadata category is used to indicate by the Part Detail entity in which the Part element has a why a media content version was created. For example, a colorized version of an originally black-and-white movie value of “1”. The part aired on the second night is identified can be indicated using the Version Detail metadata category. by the Part Detail entity in which the Part element has a An example entity within the Version Detail metadata cat value of “2. egory has the following elements: The Total Parts element stores an integer indicating the 10 total number of parts associated with the piece of media Version Reason MSI content. For example, for a movie aired in two parts over Version Description Date Time two nights, the Total Parts element has a value of “2. Entry Index Language Entry Tag The Entry Index and Entry Tag elements are described 15 above and indicate that a metadata provider may submit The Version Reason element stores a value that indicates metadata describing media content that is divided into parts the nature of the media content version. An example selec in multiple ways. For example, the same movie may be aired tion list of values for the Version Reason element is given in over two nights by one television station and aired over three Table 2. nights by another television station. In this example, one set TABLE 2 Unknown Edited For Violence Resolution Increase Original Edited For Duration Colorized Edited For Language Updated Language Dub Edited For Nudity Aspect Ratio Change Subtitle Change Edited For Adult Content Resolution Reduction Other

The Version Description element stores additional infor of Part Detail entities may describe the movie as having two mation about the media content version. This element can be 30 parts, while another set of Part Detail entities may describe used to provide addition information as to the reason for the the movie as having three parts. In a composite description, creation of the version, for example, if the Version Reason the Entry Index and Entry Tag elements may differentiate element contains the value “Other, the Version Description between various opinions as to the part details, the opinions element can be used to specify the other reason. As another example, the Version Description element may be used to 35 submitted by multiple metadata providers indicate why the version was created, for example, as a The MSI element is described above and identifies the special re-release of a movie to mark the 50' anniversary of metadata provider. the original release of the movie. The Date Time element is described above and indicates The Entry Index and Entry Tag elements are described the date and time at which the data was received. above and indicate that a metadata provider may submit 40 metadata associated with multiple versions of the media Associated People 604 content, or a composite description may indicate various FIG. 8 illustrates an example associated people metadata opinions as to a reson or description of a version if the media category. Associated people 604 includes data that describes content, the opinions Submitted by multiple metadata pro people who are associated with the media content. An viders. 45 example associated people metadata category is person. The MSI element is described above and identifies the Person metadata provider. The Date Time element is described above and indicates The Person metadata category is used to store data the date and time at which the data was received. identifying persons who are associated with the media The Language element is described above and identifies 50 content. For example, with reference to a movie, actors, the language associated with the text in the Version Descrip directors, Screenwriters, and producers are all persons asso tion element. ciated with the movie. An example entity within the Person Part Detail metadata category has the following elements: The Part Detail metadata category is used to indicate 55 multiple parts of the media content. For example, television stations often broadcast a long movie over two nights. The two parts of the media content, each aired separately, are MPI Entry Tag identified by separate entities in the Part Detail metadata Name MSI category. An example entity within the Part Detail metadata 60 Person Role Date Time category has the following elements: Character Language Entry Index Part Entry Tag Total Parts MSI Entry Index Date Time 65 The MPI element stores a unique identifier associated with the person. The media content description system 304 stores the MPI (media person identifier) and other data US 7,363,649 B2 15 16 associated with each person in the media person data reposi Genre Program Type tory 414, which is described with reference to FIGS. 18-19. The Genre Program Type metadata category is used to The Name element is used to store the name of the person. The Person Role element is used to store an indicator that store data that identifies a media type or category associated identifies how the person is associated with the media 5 with the media content. An example entity within the Genre content. An example list of possible values for the Person Program Type metadata category has the following ele Role element is given in Table 3. ments:

TABLE 3 Unknown Sound Engineer Set Designer General Participant Video Engineer Set Maker Actor Camera Operator Computer Engineer Actor Group Photographer Property Logistics Director Cinematographer Stunts Assistant Director Lighting Special Effects Producer Host Sound Effects Executive Producer Anchor Assistant Editor Editor News Reporter Assistant Camera Operator Script Writer Interviewer Makeup Lyrics Writer Performer Costumes Music Composer Narrator Title Designer Music Artist Dancer Marketing Music Artist Keyboard Animator Assistant Music Artist Drummer Graphics Artist Staff Music Artist Guitarist Casting Distributor Music Artist Lead Singer Continuity Publisher Music Artist Backing Singer Art Director Aggregator Music Band Technical Direction Contestant Manager Production Manager Subject Of Program Music Supervisor Production Designer Other Production Assistant

30 The Character element is used to store the name of a character played by the identified person. For example, for the television series, “Frasier, the Person entity identifying Value Entry Tag Date Time the actor, Kelsey Grammer, may have the value “Dr. Frasier Entry Index MSI Crane' in the Character element. 35 The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit The Value element stores an indicator of a category or metadata describing a list of cast members associated with genre associated with the media content. An example list of the media content. A second and less common use of the values for populating the Value element is given in Table 4. Entry Index and Entry Tag elements is to identify priorities 40 among a plurality of associations that one person may have TABLE 4 with the media. For example, a particular actor may play Movie Drama more than one character in a movie, or an actor may also be None the director of a movie. In a composite description, the Entry Unknown Index and Entry Tag elements may differentiate between 45 General News And Current Affairs - various opinions as to a role or character played by a person, News And Current Affairs Magazine the opinions submitted by multiple metadata providers News And Current Affairs - Legal And Investigative Journalism The MSI element is described above and identifies the News And Current Affairs - Showbiz And Personality News News And Current Affairs - Special News Program metadata provider. The Date Time element is described News And Current Affairs - Past News Roundup above and indicates the date and time at which the data was News And Current Affairs - News About Education received. The Language element is described above and News And Current Affairs - Service Information identifies the language associated with the character ele News And Current Affairs - General News And Current Affairs - Daily News ment. For example, a character may have one name in News And Current Affairs - Traffic France, and the same character may have a different name in 55 News And Current Affairs - Weather America. News And Current Affairs - Politics News And Current Affairs - Economy Genre Data 606 News And Current Affairs - Business FIG. 9 illustrates example genre data categories. Genre News And Current Affairs - Financial data 606 includes metadata categories that describes a genre News And Current Affairs - Sports News News And Current Affairs - Social Report or category in which the media content may be included. 60 News And Current Affairs - Press Roundup Example genre data categories include genre program type, News And Current Affairs - Other genre degrees, genre intent, target audience, year set, era set, Informational - Genera location portrayed, from the book, degree of true story, Informational - Money Advice Informational - Legal Magazine degree of animated 2D. degree of animated 3D. degree of Informational - Health Magazine puppet characters, degree of international event, degree of 65 Informational - Personal Problems Magazine Sophistication, music genre, genre photo or home movie, Informational - Sex Information Magazine and format. US 7,363,649 B2 17 18

TABLE 4-continued TABLE 4-continued informational - Shopping Leisure - Dining Out informational - Interview Leisure - Boating informational - Lecture 5 Leisure - Motoring informational - Schools Program Leisure - Motorcycling informational - Speech Or Presentation Leisure - Cycling informational - Ethnic Leisure - Hiking And Camping informational - Language Learning Leisure - Outdoors informational - Other Leisure - Keep Fit Documentary - General 10 Leisure - Fashion Documentary - Science Leisure - Computing Documentary - Technology Leisure - Video Gaming Documentary - Engineering And Construction Leisure - Audio Video Equipment Documentary - Transport Leisure - Pets Documentary - Historical Leisure - Antique Collecting Documentary - Medical 15 Leisure - Painting And Sculpture Learning Documentary - Environment Leisure - Fishing Learning Documentary - Space Leisure - Musical Instruments Learning Documentary - Undersea Leisure - Skiing Learning Documentary - Social Science Leisure - Snowboarding Learning Documentary - Psychology Leisure - Boardsailing Learning Documentary - Sex Leisure - Golf Learning Documentary - Legal Leisure - Chess Learning Documentary - Music Leisure - Sports. Other Learning Documentary - Biography Leisure - Other Documentary - Business And Specific Industries Religions And Philosophies - General Documentary - Nature Religions And Philosophies - Christianity Documentary - Religions Religions And Philosophies - Judaism Documentary - Culture And Traditions 25 Religions And Philosophies - Buddhism Documentary - Foreign Countries Religions And Philosophies - Islam Documentary - Expeditions And Adventure Religions And Philosophies - Hinduism Documentary - Politics Religions And Philosophies - Agnosticism Documentary - Past News Events Religions And Philosophies - Atheism Documentary - Showbiz Religions And Philosophies - Humanism Documentary - Other 30 Religions And Philosophies - Communism General Entertainment And Comedy - General Religions And Philosophies - Socialism General Entertainment And Comedy - SitCom Religions. And Philosophies - Capitalism General Entertainment And Comedy - Standup Comedy Religions And Philosophies - Libertarianism General Entertainment And Comedy - Comedy Sketch Show Religions And Philosophies - Republicanism General Entertainment And Comedy - Comedy News Parody Religions And Philosophies - Other General Entertainment And Comedy - Reality Show 35 Events - General General Entertainment And Comedy - Variety Show Events - Nationally Significant Event General Entertainment And Comedy - Courtroom Events - Celebrity Event General Entertainment And Comedy - Celebrity Chat Show Events - Non Celebrity Event General Entertainment And Comedy - Public Talk Show Events - Military Event General Entertainment And Comedy - Quiz Game Show Events - Political Event General Entertainment And Comedy - Contest Show 40 Events - Ceremonies General Entertainment And Comedy - Bloopers Events - Festivals General Entertainment And Comedy - Real Life Drama Events - Other General Entertainment And Comedy - Surprise Show Sport - General General Entertainment And Comedy - Comedy Entertainer Show port - Acrobatics General Entertainment And Comedy - Musical Entertainer Show port - Aeronautics General Entertainment And Comedy - Magic port - Aikido General Entertainment And Comedy - Panel Quiz Show 45 port - American Football General Entertainment And Comedy - Other port - Archery Arts And Culture - General port - Athletics Arts And Culture - Pop Music port - Badminton Arts And Culture - Classical Music port - Bandy Arts And Culture - Old Time Music Hall port - Baseball Arts And Culture - Paintings And Sculpture 50 port - Basketball Arts And Culture - Theater And Performing Arts port - Biathlon Arts And Culture - Ballet port - Billiards Arts And Culture - Opera port - Board Sailing Arts And Culture - New Media port - Bobsleighing And Tobogganing Arts And Culture - Traditional port - Body Building Arts And Culture - Literature And Poetry 55 port - Boule And Bowls Arts And Culture - Languages port - Bowling Arts And Culture - Architecture port - Boxing Arts And Culture - Cinema Reviews port - Canoeing Arts And Culture - TV And Radio Reviews port - Casting Arts And Culture - Other port - Chess Leisure - General port - Computer Gaming Leisure - Diy Home 60 port - Cricket Leisure - Diy Car port - Croquet Leisure - Cookery port - Curling Leisure - Sewing port - Cycling Leisure - Home Making port - Dance Sport Leisure - Gardening port - Darts Leisure - Handicrafts 65 port - Diving And Subaquatics Leisure - Travel And Tourism port - Dog Racing US 7,363,649 B2 19 20

TABLE 4-continued TABLE 4-continued port - Equestrian Excluding Horse Racing vertising - Household Furnishings port - Faustball vertising - Audio Video Equipment port - Fencing vertising - Computers And Cameras port - Fishing vertising - Cars Bikes And Boats port - Flying Disc vertising - Pets vertising - Clothes And Shoes port - Gymnastics vertising - Entertainment port - Handball vertising - Travel And Hotels port - Hockey 10 vertising - Restaurants port - Horse Racing vertising - Shopping Stores port - Ice Hockey vertising - Financial Services And RealEstate port - Jai Alai vertising - Publications port - Judo vertising - Public Utilities port - Jujitsu vertising - Company Publicity port - Karate 15 vertising - Government Information port - Korfball vertising - Political port - LacroSSe vertising - TV Program Trailers port - Luge vertising - Direct Sell port - Maccabi vertising - Infomercial port - Marathon vertising - Other port - Modern Pentathlon udio - General port - Motor Boating udio - Music Track port - Motorcycling udio - Book port - Motor Racing Cars udio - Talk port - Mountaineering Au io - Music And Talk port - Netball Audio - Other port - Orienteering And Hiking PhotoOromeMovie - General port - Polo 25 PhotoOromeMovie - Professional Of Place port - Power Lifting PhotoCorHomeMovie - Professional Of People port - Racquetball And Squash PhotoOromeMovie - Professional Of Event port - Roller Skating PhotoOromeMovie - Professional Of Document port - Rowing PhotoCorHomeMovie - Professional Graphic port - Rugby PhotoOromeMovie - Amateur Of Place port - Running 30 PhotoCorHomeMovie - Amateur Of People port - Shooting PhotoOromeMovie - Amateur Of Event port - Skating Ice PhotoCrHomeMovie - Amateur Of Document port - Skibob PhotoCorHomeMovie - Amateur Graphic port - Skiing PhotoOromeMovie - Other port - Sky Diving Other port - Sleddog 35 port - Snooker port - Snowboarding The Entry Index and Entry Tag elements are described port - Soccer Football port - Soft Tennis above and indicate that a metadata provider may submit port - Softball metadata that categorizes the media content multiple ways, port - Sombo or a composite description may indicate various opinions as port - Sumo Wrestling 40 to the genre program type, the opinions Submitted by mul port - Surfing port - Swimming And Diving tiple metadata providers. For example, a weekend sports port - Table Tennis show may have three different segments, one covering port - Taekwondo football, one covering hockey, and one covering downhill port - Tennis skiing. The media may be categorized as Sport - General, port - Track And Element 45 port - Trampoline Sport - American Football, Sport - Hockey, or Sport - port - Triathlon Skiing. port - Tug Of War The MSI element is described above and identifies the port - Volleyball metadata provider. port - Water Polo The Date Time element is described above and indicates port - Water Skiing 50 port - Weight Lifting the date and time at which the data was received. port - Wrestling Genre Degrees port - Yachting port - Other Ball Game The Genre Degrees metadata category is used to further port - Other Combative Sport categorize media that is specified in the Genre Program Type port - Other Martial Sports 55 metadata category as "MovieOrama'. An example entity port - Other Oriental Sports within the Genre Degrees metadata category has the follow port - Other Team Sport port - Other Water Sport ing elements: port - Other Winter Sport port - Other Advertising - General Advertising - Medicine And Sanitary Products 60 Action Death Political Intrigue Advertising - Toiletries Adventure Mystery Romance Advertising - Cosmetics Horror Police Involvement Erotica Advertising - Alcohol And Tobacco Comedy Thriller Science Fiction Advertising - Soft Drinks Period Setting War Interest Soap Advertising - Sweets And Puddings Lives Drama Epic Production Entry Index Advertising - Foods 65 Sports Interest Fantasy Folklore Entry Tag Advertising - Household Cleaning And Painting Animal Interest Musical MSI US 7,363,649 B2 22 metadata that categorizes the intent of the media content -continued multiple ways, or a composite description may indicate Medical Interest Western Date Time various opinions as to the intent of the media content, the Legal Interest Monsters opinions submitted by multiple metadata providers. For Religious Interest Teenage College example, a second categorization of the television infomer Historical Interest Ethnic Interest cial mentioned above may have a value of 100 in the Sell element and the other element values set to 0. The Action, Adventure, Horror, Comedy, Death, Mystery, The MSI element is described above and identifies the Police Involvement, Thriller, Political Intrigue, Romance, metadata provider. The Date Time element is described Erotica, Science Fiction, Period Setting, Lives Drama, 10 above and indicates the date and time at which the data was Sports Interest, Animal Interest, Medical Interest, Legal received. Interest, Religious Interest, Historical Interest, War Interest, Target Audience Epic Production, Fantasy Folklore, Musical, Western, Mon The Target Audience metadata category is used to store sters, Teenage College, Ethnic Interest, and Soap elements data that describes characteristics of audiences targeted by each store an integer number that indicates a percentage 15 the media content. An example entity within the Target value that indicates how well the media content falls into the Audience metadata category has the following elements: respective category. In an exemplary system, the sum of the values in these elements is 100. For example, to indicate that the movie “Ausin Powers' is mostly a comedy, but also Gender Education Entry Index includes elements of romance and Science Fiction, the Age Ethnic Origin Entry Tag Comedy, Romance, and Science Fiction elements may be Marital Status Religion MSI assigned values 70, 25, and 5, respectively. Household Occupation Date Time The Entry Index and Entry Tag elements are described Annual Income above and indicate that a metadata provider may submit metadata that categorizes the media content multiple ways, 25 or a composite description may indicate various opinions as The Gender element stores an indicator of the gender that to the categorization of the media content, the opinions the media content targets. Example values for populating the submitted by multiple metadata providers. For example, the Gender element may include: Unknown, All, Male, Female, movie “Ausin Powers' may be additionally categorized as Male Homosexual, Female Homosexual, and Other. 90% comedy and 10% romance. 30 The Age element stores an indicator of the age or age The MSI element is described above and identifies the group that the media content targets. Example values for metadata provider. populating the Age element may include: Unknown, All. The Date Time element is described above and indicates 0-5,6-12, 13-19, 20-34, 35-54, and 55+. the date and time at which the data was received. The Marital Status element stores an indicator of the Genre Intent 35 marital status of members of the audience targeted by the The Genre Intent metadata category is used to store data media content. Example values for populating the Marital that describes the purpose of the media content (e.g., why Status element may include: Unknown, All, Single, Mar the program was made or what the program is Supposed to ried NO Children, Married With Children, and Single do). An example entity within the Genre Intent metadata With Children. category has the following elements: 40 The Household Annual Income element stores an indica tor of the average household income of members of the audience targeted by the media content. Example values for Education Enrich Or Involve MSI populating the Household Annual Income element may Entertainment Sell Date Time 45 include: Unknown, All, 0-34KS, 35-69KS, 70-139KS, and News Entry Index 140+KS. Information Entry Tag The Education element stores an indicator of the average level of education of members of the audience targeted by The Education, Entertainment, News, Information, Enrich the media content. Example values for populating the Edu Or Involve, and Sell elements each store an integer number 50 cation element may include: Unknown, All, Low, Average, that indicates a percentage value to which the purpose of the and High. media falls into the respective category. In an exemplary The Ethnic Origin element stores an indicator of the system, the sum of the values in these elements is 100. For ethnic origin of members of the audience targeted by the example, a television sitcom may be associated with a value media content. Table 5 is a list of example values for of 100 in the Entertainment element, while a television populating the Ethnic Origin element.

TABLE 5 Unknown Eastern European Indian Asian Original Peoples All Latino Far Eastern Other Western European African Arabic infomercial may have a value of 80 in the Sell element and The Religion element stores an indicator of the religion of a value of 20 in the Information element. 65 members of the audience targeted by the media content. The Entry Index and Entry Tag elements are described Table 6 is a list of example values for populating the above and indicate that a metadata provider may submit Religion element. US 7,363,649 B2 24 The Value element stores an indicator of an era portrayed TABLE 6 in the media content. Table 8 is a list of example ranges of Unknown Christian Buddhist Hindu Atheist values for populating the Value element. All Jewish Islamic Agnostic Other TABLE 8 Unknown SOO-1SOO 1960-2OOO The Occupation element stores an indicator of the occu None 1500-1800 Today Approx pation of members of the audience targeted by the media Before 1800-1900 Up To 100 Years. In The Future content. Table 7 is a list of example values for populating the 1OOBC Occupation element. 10 1OOBC- 1900-1960 More Than 100 Years. In The Future SOOAD TABLE 7 Unknown All Office Worker Technologist Manager The Entry Index and Entry Tag elements are described Not Employed Crafts Or Skill Worker Other above and indicate that a metadata provider may submit Manual Worker Profession Worker 15 metadata that indicates a plurality of eras portrayed within the media content, or a composite description may indicate The Entry Index and Entry Tag elements are described various opinions as to the era in which the media content is above and indicate that a metadata provider may submit set, the opinions Submitted by multiple metadata providers. metadata that categorizes multiple target audiences, or a For example, a movie about time travel may have several composite description may indicate various opinions as to Era Set entities, each with a Value element representing a the target audience, the opinions Submitted by multiple different era portrayed in the movie. metadata providers. For example, the media content may be The MSI element is described above and identifies the targeted at individuals in a plurality of occupations, genders, metadata provider. or income levels. The Date Time element is described above and indicates The MSI element is described above and identifies the 25 the date and time at which the data was received. metadata provider. Location Portrayed The Date Time element is described above and indicates The Location Portrayed metadata category is used to store the date and time at which the data was received. data indicating a location portrayed in the media content. An Year Set example entity within the Location Portrayed metadata The Year Set metadata category is used to store data 30 category has the following elements: indicating a year portrayed within the media content. An example entity within the Year Set metadata category has the following elements: Astronomical City MSI 35 Location Location Portrayed Detail Date Time Country Entry Index Language State Entry Tag Value MSI Entry Index Date Time Entry Tag The Astronomical Location element stores a name or 40 description of an astronomical location (e.g., Earth, the The Value element stores a number that indicates a year moon, Mars, another galaxy far away, etc.) portrayed in the portrayed in the media content. For example, a value of media content. “1942 may be associated with a movie set in 1942. The Country element stores a value that indicates a The Entry Index and Entry Tag elements are described country portrayed in the media content. A list of possible above and indicate that a metadata provider may submit 45 values for populating the Country element is given below, in metadata that indicates a plurality of years portrayed within Table 9. the media content, or a composite description may indicate various opinions as to a year portrayed, the opinions Sub The State element stores a value that indicates a state mitted by multiple metadata providers. For example, a portrayed in the media content. The value may be the name movie that spans five years of an individuals life may have 50 of a state, such as “Washington', or an abbreviation asso five Year Set Entries entities, one for each year portrayed. ciated with the state, such as “WA'. The MSI element is described above and identifies the The City element stores the name of a city portrayed in the metadata provider. media content. The Date Time element is described above and indicates The Location Portrayed Detail element is used to store the date and time at which the data was received. 55 additional details that describe a location portrayed in the Era Set media content. For example, the Location Portrayed Detail The Era Set metadata category is used to store data element may contain the value, “Alcatraz Island in the associated with an era portrayed within the media content. middle of San Francisco Bay,’ in a Location Portrayed An example entity within an Era Set Entries metadata element associated with the movie, “Escape from Alcatraz’. category has the following elements: 60 The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit data describing multiple locations portrayed in the media content, or a composite description may indicate various opinions as Value MSI Entry Index Date Time to the location portrayed, the opinions submitted by multiple Entry Tag 65 metadata providers. The MSI element is described above and identifies the metadata provider. US 7,363,649 B2 25 26 The Date Time element is described above and indicates the date and time at which the data was received. TABLE 9-continued The Language element is described above and identifies FJ Fiji the language associated with the value of the Location FK Falkland Islands FM Micronesia Portrayed Detail element. FO Faroe Islands FR France TABLE 9 FX France (European Territory) Unknown None 10 E Georgia All F French Guyana Not Applicable H. Ghana AD Andorra, Principality of Gibraltar AE United Arab Emirates L. Greenland AF Afghanistan, Islamic State of M. Gambia AG Antigua and Barbuda 15 Gu l(8. AI Anguilla Guadeloupe (French) AL Albania Equatorial Guinea AM Armenia Greece AN Netherlands Antilles S.Georgia and S.Sandwich Isls. AO Angola T Guatemala AQ Antarctica Guam (USA) AR Argentina Guinea Bissau AS American Samoa Guyana AT Austria K Hong Kong U Australia HM Heard and McDonald Islands W Aruba HN Honduras AZ Azerbaidjan HR Croatia Bosnia-Herzegovina 25 HT Haiti Barbados HU Hungary D Bangladesh Indonesia Belgium reland Burkina Faso ONP-ED srael Bulgaria India : Bahrain 30 British Indian Ocean Territory Burundi raq Benin 8 Bermuda celand Brunei Darussalam - taly Bolivia Jamaica Brazil 35 Jordan Bahamas apan - Bhutan Kenya BV Bouvet Island Kyrgyz Republic (Kyrgyzstan) BW Botswana KH Cambodia, Kingdom of BY Belarus KI Kiribati BZ Belize KM Comoros CA Canada 40 KN Saint Kitts and Nevis Anguilla CC Cocos (Keeling) Islands KP North Korea CF Central African Republic KR South Korea CD Congo, The Democratic Republic of the KW Kuwait CG Congo KY Cayman Islands CH Switzerland KZ Kazakhstan CI Ivory Coast (Cote D'Ivoire) 45 LA Laos CK Cook Islands LB Lebanon CL Chile LC. Saint Lucia CM Cameroon LI Liechtenstein CN China LK Sri Lanka CO Colombia LR Liberia CR Costa Rica 50 LS. Lesotho CS Former Czechoslovakia LT Lithuania CU Cuba LU Luxembourg CV Cape Verde LV Latvia CX Christmas Island LY Libya CY Cyprus MA Morocco CZ Czech Republic 55 MC Monaco DE Germany MD Moldavia DJ Djibouti MG Madagascar DK Denmark MH Marshall Islands DM Dominica MK Macedonia DO Dominican Republic ML. Mali DZ Algeria MM Myanmar EC Ecuador 60 MN Mongolia EE Estonia MO Macau EG Egypt MP Northern Mariana Islands EH Western Sahara MQ Martinique (French) ER. Eritrea MR Mauritania ES Spain MS Montserrat ET Ethiopia 65 MT Malta FI Finland MU Mauritius US 7,363,649 B2 27 28

TABLE 9-continued TABLE 9-continued MV Maldives UZ Uzbekistan MW Malawi VA Holy See (Vatican City State) MX Mexico VC Saint Vincent and Grenadines MY Malaysia VE Venezuela MZ Mozambique VG Virgin Islands (British) NA. Namibia VI Virgin Islands (USA) NC New Caledonia (French) VN Vietnam NE Niger VU Vanuatu NF Norfolk Island 10 WF Wallis and Futuna Islands NG Nigeria WS Samoa NI Nicaragua YE Yemen NL Netherlands YT Mayotte NO Norway YU Yugoslavia NP Nepal ZA South Africa NR Nauru 15 ZM Zambia NT Neutral Zone ZR Zaire NU Niue ZW Zimbabwe NZ New Zealand OM Oman PA Panama From The Book PE Peru PF Polynesia (French) The From The Book metadata category is used to store PG Papua New Guinea data that identifies a book that was the source of inspiration PH Philippines for the media content. For example, the movie, "A Walk to PK Pakistan Remember, is based on a book by the same name, by PL Poland PM Saint Pierre and Miquelon author, Nicholas Sparks. An example entity within the From PN. Pitcairn Island 25 The Book metadata category has the following elements: PR Puerto Rico PT Portugal PW Palau PY Paraguay Book Title Entry Index MSI QA Qatar Book Author Entry Tag Date Time RE. Reunion (French) 30 ROO Romania RU Russian Federation The Book Title element stores the title of the book on RW Rwanda which the media content is based. S A Saudi Arabia S B Solomon Islands The Book Author element stores the name of the author of S C Seychelles 35 the book identified in the Book Title element. S D Sudan S E. Sweden The Entry Index and Entry Tag elements are described S G Singapore above and indicate that a metadata provider may submit data S H Saint Helena identifying multiple books associated with the media con S Slovenia tent, or a composite description may indicate various opin S Svalbard and Jan Mayen Islands 40 S K Slovak Republic ions as to a book associated with the media content, the S L. Sierra Leone opinions submitted by multiple metadata providers. For S M. San Marino example, a movie may be based on a combination of a first S book and a second book that is a sequel to the first book. S S The MSI element is described above and identifies the S T Saint Tome (Sao Tome) and Principe 45 metadata provider. SU Former USSR The Date Time element is described above and indicates SV El Salvador the date and time at which the data was received. SY Syria SZ Swaziland Degree of True Story TC Turks and Caicos Islands The Degree Of True Story metadata category is used to TD Chad 50 TF French Southern Territories store data that identifies whether the media content portrays TG. Togo actual events. An example entity within the Degree Of True TH Thailand Story metadata category has the following elements: TJ Tadjikistan TK Tokelau TM Turkmenistan 55 TN. Tunisia Value MSI TO Tonga Entry Index Date Time TP East Timor Entry Tag TR Turkey TT Trinidad and Tobago TV Tuvalu TW Taiwan 60 The Value element stores an integer number between 0 TZ Tanzania and 100 that indicates the percentage to which the media UA Ukraine content portrays actual events. For example, when associ UG Uganda UK United Kingdom ated with media content that is a documentary, the Value UM USA Minor Outlying Islands element may store the number 100, indicating that the events 65 portrayed are 100 percent true and actual. When associated with media content may be categorized as historical fiction, meaning that the story is fictional, but based on actual US 7,363,649 B2 29 30 historical events, the Value element may store the number The Entry Index and Entry Tag elements are described 50, indicating that the events portrayed are only 50 percent above and indicate that a metadata provider may submit true and actual. multiple opinions as to the degree of three-dimensional The Entry Index and Entry Tag elements are described animation associated with the media content, or a composite above and indicate that a metadata provider may submit 5 description may indicate various opinions as to the degree of multiple opinions as to the degree of actuality associated three-dimensional animation, the opinions Submitted by with the media content, or a composite description may multiple metadata providers. indicate various opinions as to the degree of actuality, the The MSI element is described above and identifies the opinions Submitted by multiple metadata providers. metadata provider. The MSI element is described above and identifies the 10 The Date Time element is described above and indicates metadata provider. the date and time at which the data was received. The Date Time element is described above and indicates Degree of Puppet Characters the date and time at which the data was received. The Degree Of Puppet Characters metadata category is Degree Of Animated 2D used to store data that indicates the amount of puppet The Degree Of Animated 2D metadata category is used to 15 characters within the media content. An example entity store data that indicates the amount of two-dimensional within the Degree Of Puppet Characters metadata category animation contained within the media content. An example has the following elements: entity within the Degree Of Animated 2D metadata category has the following elements: 2O Value MSI Entry Index Date Time Entry Tag Value MSI Entry Index Date Time Entrytry TagTa. 25 The Value element stores an integer number between 0 and 100 that indicates the percentage to which the characters The Value element stores an integer number between 0 in the media content are puppets. For example, when asso and 100 that indicates the percentage to which the media ciated with the television series, “Sesame Street', the Value content is animated in two dimensions. For example, when element may store the number 90, indicating that 90 percent associated with the animated movie, “The Lion King', the 30 of the characters are puppets. Value element may store the number 100, indicating that the The Entry Index and Entry Tag elements are described movie is 100 percent two-dimensional animation. When above and indicate that a metadata provider may submit associated with the movie, “Mary Poppins', which is a live multiple opinions as to the degree of puppet characters action film with Small elements of two-dimensional anima associated with the media content, or a composite descrip tion, the Value element may store the number 10, indicating 35 tion may indicate various opinions as to the degree of puppet that the media content is 10 percent two-dimensional ani characters, the opinions Submitted by multiple metadata mation. providers. The Entry Index and Entry Tag elements are described The MSI element is described above and identifies the above and indicate that a metadata provider may submit metadata provider. multiple opinions as to the degree of two-dimensional 40 The Date Time element is described above and indicates animation associated with the media content, or a composite the date and time at which the data was received. description may indicate various opinions as to the degree of Degree of International Event two-dimensional animation, the opinions Submitted by mul The Degree Of International Event metadata category is tiple metadata providers. used to store data identifying the degree to which the media The MSI element is described above and identifies the 45 content pertains to an international event. An example entity metadata provider. within the Degree Of International Event metadata category The Date Time element is described above and indicates has the following elements: the date and time at which the data was received. Degree Of Animated 3D 50 The Degree Of Animated 3D metadata category is used to Value MSI store data that indicates the amount of three-dimensional Entry Index Date Time animation contained within the media content. An example Entry Tag entity within the Degree Of Animated 3D metadata category has the following elements: 55 The Value element stores an integer number between 0 and 100 that indicates the percentage to which the media content pertains to an international event. For example, a Value MSI high percentage of International Event may be associated Entry Index Date Time with media coverage of the Olymics is a high percentage of Entrytry TagTa. 60 International Event. The Entry Index and Entry Tag elements are described The Value element stores an integer number between 0 above and indicate that a metadata provider may submit and 100 that indicates the percentage to which the media multiple opinions as to the degree to which the media content is animated in three dimensions. For example, when content pertains to an international event, or a composite associated with the animated movie, "Toy Story’, the Value 65 description may indicate various opinions as to the degree to element may store the number 100, indicating that the movie which the media content pertains to an international event, is 100 percent three-dimensional animation. the opinions submitted by multiple metadata providers. US 7,363,649 B2 31 32 The MSI element is described above and identifies the metadata provider. TABLE 10-continued The Date Time element is described above and indicates the date and time at which the data was received. Austic Degree of Sophistication 5 Afropop The Degree Of Sophistication metadata category is used Alternative to store data how Sophisticated the media content is. An Alistick example entity within the Degree Of Sophistication meta- Atonalel data category has the following elements: Avantgarde 10 Ballad Barbershop Quartet Baroque Value MSI Bass Entry Index Date Time Bebop Entry Tag Bhangra 15 Big Band Bluegrass The Value element stores an integer number between 0 Bus Bass and 100 that indicates the level to which the media content Bop is Sophisticated. For example, a high degree of Sophistica- Brazilian tion may be associated with an opera, while a low degree of 20 BritPop Sophistication may be associated with a program Such as abat “Gladiators. CE The Entry Index and Entry Tag elements are described Chamber Music above and indicate that a metadata provider may submit Chanson multiple opinions as to the degree of Sophistication associ- 2s Re Blues ated with the media content, or a composite description may Childrens indicate various opinions as to the degree of Sophistication, Chinese the opinions submitted by multiple metadata providers. his The MSI element is described above and identifies the ChristianOS Rap metadata provider. 30 Christian Rock The Date Time element is described above and indicates Classic Rock the date and time at which the data was received. Classical Light 8SS1Cal. 1 Genre Music Club The Genre Music metadata category is used to store data Comedy that identifies a category associated with media content that 35 Computer Music - 0 Contemporary Christian is related to music. An example entity within the Genre Contemporary Classical Music metadata category has the following elements: Country Country And Western Creole Crossover 40 Value MSI Cult Entry Index Date Time R Hall Entry Ta. alCe Eda try Tag DarkWave Death Metal The Value element stores the name of a music category or as Ric Blues genre that applies to the media content. Example music 5tant categories are listed in Table 10, below. Dixieland The Entry Index and Entry Tag elements are described Dream above and indicate that a metadata provider may submit By solo multiple opinions as to the music genre associated with the so Duetl media content, or a composite description may indicate Early Music various opinions as to the music genre, the opinions Sub- Easy Listening mitted by multiple metadata providers. Estonic The MSI element is described above and identifies the Euro-HouseC metadata provider. 55 Euro-Techno The Date Time element is described above and indicates Eurodance the date and time at which the data was received. Experimenta Fast Fusion Flamenco TABLE 10 Folk Folklore Unknown 60 Folk Rock None Free Jazz Not Applicable Freestyle Various Fun A Capella Funk Acid Fusion Acid Jazz 65 Game Acid Punk Gangsta US 7,363,649 B2 33 34

TABLE 10-continued TABLE 10-continued Garage Reggae Glam Rock Religious Glee Club Renaissance Glide Retro Gospel Revival Gothic Rock Rhythm And Blues Gothic Rhythmic Soul Gregorian Chant Riot Girl Grunge 10 Rock Hard Bop Rock And Roll Hard Rock Rockabilly Hardcore Romantic Classical Hip-Hop Sacred House Samba Humour 15 Satire Hymns Shoegazers bient ShowTunes improvisatory Sing-Along Indian Classical Ska. indie Slow Jam industrial Slow Rock instrumental Smooch instrumental Pop Soft Rock instrumental Rock Sonata apanese Soul Sound Clip aZZ-Funk SoundTrack aZZ Modern 25 Southern Rock aZZ Traditional Space ungle Speech ust Dance Spirituals Karaoke Backing StraightEdge Kids Surf Rock Latin 30 Swing Latin Jazz Symphonic Listen Symphonic Rock Lo-Fi Symphony Lounge Talking Blues March Talking Book Marching Band 35 Tango Meditative Techno Metal Techno-Industrial Microtonal Teeny-Bop Minimalism Tejano Modal Tex-Mex Modern Classical Tin Pan Alley Modern Rock 40 Top 40 Motown Trailer Musical Trance Muzak Tribal National Folk Trip-Hop Native American Twee Pop New Age 45 Underground New Music Vaudeville New Wave Vocal Noise Wild Nostalgia Womens Music Novelty World Beat Old Dance 50 World Fusion Old School World Pop Oldies Zydeco Opera Other Operetta Polka Pop 55 Genre Photo Or Home Movie Pop-Folk Pop-Funk The Genre Photo Or Home Movie metadata category is Porn Groove used to store data that identifies a genre or category that may Power Ballad be associated with the media content. An example entity Pranks Primus within the Genre Photo Or Home Movie metadata category Progressive Rock 60 has the following elements: Psychadelic Psychedelic Rock Punk R And B Value MSI Ragtime Entry Index Date Time Rap 65 Entry Tag Rave US 7,363,649 B2 35 36 The Value element stores the name of a category or genre The MSI element is described above and identifies the that applies to media content that includes photos or home metadata provider. movies. Example photo or home movie categories are listed The Date Time element is described above and indicates in Table 11, below. the date and time at which the data was received.

TABLE 12 Unknown Video - Kids Or. Other Cartoon None Video - How-To Or Educational Video - Unspecified Type Video - Religious Local Access Or Paid Video - Movie Cinema Released Video - Specials Video - Drama Mini Series Or TV Movie Video - Other Video - SitCom Video - Amateur Video - Soap Audio - Unspecified Type Video - Sports Audio - Song Track Video - Games Or Reality Audio - Talking Book Video - Talk Show. Or Non SitCom Comedy Audio - Amateur Recording Video - News Or Current Affairs Audio - Other Video - Documentary Photo - Unspecified Type Photo - Amateur Collection - Video Photo - Professional Collection - Audio Photo - Other Collection - Photo Advert Collection - Other Collection - Unspecified Type

The Entry Index and Entry Tag elements are described 25 Content Descriptions 608 above and indicate that a metadata provider may submit FIG. 10 illustrates example content description catego multiple opinions as to the genre or category associated with ries. Content descriptions 608 includes metadata categories the photo or home movie media content, or a composite that describe the media content. Example content descrip description may indicate various opinions as to the genre or tion categories include plot one line abstract, plot long category, the opinions Submitted by multiple metadata pro 30 abstract, keywords, content note, availability date, and con viders. tent duration. The MSI element is described above and identifies the Plot One Line Abstract metadata provider. The Plot One Line Abstract metadata category is used to The Date Time element is described above and indicates store a short description of the plot associated with the media the date and time at which the data was received. 35 content. An example entity within the Plot One Line Abstract metadata category has the following elements: TABLE 11 Unknown Various At Home From Relatives None Vacation Purchased From Friends Value MSI Not Applicable Kids Event Famous Art Other 40 Entry Index Date Time Entry Tag Language Format The Value element stores a one-line textual description of The Format metadata category is used to store data that the plot of the media content. For example, for the Frasier identifies a format associated with the media content. For 45 episode titled, “My Coffee with Niles', the Value element mat can be thought of as a broad assessment of genre. In one may contain, “After a year in Seattle, Niles poses Frasier implementation, the format metadata can be used by an EPG with a deceptively simple question: “Are you happy? at one application to determine, for example, colors to display in of their regular meetings at Cafe Nervosa.” association with information about the media content in an The Entry Index and Entry Tag elements are described EPG grid. An example entity within the Format metadata 50 above and indicate that a metadata provider may submit category has the following elements: more than one short plot description associated with the media content, or a composite description may indicate various descriptions Submitted by multiple metadata provid Value MSI CS. 55 Entry Index Date Time The MSI element is described above and identifies the Entry Tag metadata provider. The Date Time element is described above and indicates The Value element stores an identifier of a format asso the date and time at which the data was received. ciated with the media content. Example format identifiers 60 The Language element is described above and identifies are listed in Table 12, below. the language in which the plot is described in the Value The Entry Index and Entry Tag elements are described element. above and indicate that a metadata provider may submit Plot Long Abstract more than one opinion as to the format associated with the The Plot Long Abstract metadata category is used to store media content, or a composite description may indicate 65 a longer description of the plot associated with the media various opinions as to the format, the opinions Submitted by content. An example entity within the Plot Long Abstract multiple metadata providers. metadata category has the following elements: US 7,363,649 B2

Value MSI Value Entry Tag Date Time Entry Index Date Time Entry Index MSI Language Entry Tag Language The Value element stores any text associated with the The Value element stores a textual description of the plot piece of media content that the user of the schema wants. of the media content. For example, for the Frasier episode The Entry Index and Entry Tag elements are described titled, “My Coffee with Niles', the Value element may 10 above and indicate that a metadata provider may submit contain, “Frasier meets Niles for a coffee at Cafe Nervosa, more than one content note associated with the media but they can’t find a table inside the busy cafe so decide to content, or a composite description may indicate various take a table outside. Niles then poses a deceptively simple content notes Submitted by multiple metadata providers. question “Are you happy? after Frasier reminds Niles that The MSI element is described above and identifies the it is a year since he moved to Seattle. However, the discus 15 metadata provider. sion gets repeatedly interrupted by various friends, family The Date Time element is described above and indicates members, and other discussions. Finally, as the waitress who the date and time at which the data was received. has been increasingly aggravated by Frasier's quest for the The Language element is described above and identifies perfect cup of coffee, asks “Now are you finally happy”, and the language in which the content note in the Value element Frasier says that he is.” is written. The Entry Index and Entry Tag elements are described Availability Date above and indicate that a metadata provider may submit The Availability Date metadata category is used to store more than one plot description associated with the media data that indicates when the media content was or will be content, or a composite description may indicate various made available. An example entity within the Availability descriptions submitted by multiple metadata providers. 25 Date metadata category has the following elements: The MSI element is described above and identifies the metadata provider. The Date Time element is described above and indicates the date and time at which the data was received. Value Entry Tag Date Time Entry Index MSI The Language element is described above and identifies 30 the language in which the plot is described in the Value element. The Value element stores a character string that indicates Keywords when the media content was or will be made available. The The Keywords metadata category is used to store key availability date may be a specific date, such as Feb. 10, words associated with the media content. An example entity 35 2001, or it may be a general time period, such as Fall 1997. within the Keywords metadata category has the following The availability date may also be a future date to indicate elements: media content that is not yet available. The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit 40 multiple opinions as to the availability date associated with Value MSI the media content, or a composite description may indicate Entry Index Date Time various opinions as to the availability date, the opinions Entry Tag Language submitted by multiple metadata providers. For example, metadata providers may have differing opinions about when The Value element stores a keyword or phrase associated 45 the piece of media content was actually created, or the media with the media content. For example, for the Frasier episode content may be available on different dates in different titled, “My Coffee with Niles', keywords may include: geographic areas. The EntryIndex element provides a way of coffee, happy, and “one year. indicating the various opinions. The Entry Index and Entry Tag elements are described The MSI element is described above and identifies the above and indicate that a metadata provider may submit 50 metadata provider. more than one keyword or phrase associated with the media The Date Time element is described above and indicates content, or a composite description may indicate various the date and time at which the data was received. keywords submitted by multiple metadata providers. Content Duration The MSI element is described above and identifies the The Content Duration metadata category is used to store metadata provider. 55 data associated with the length of the media content. An The Date Time element is described above and indicates example entity within the Content Duration metadata cat the date and time at which the data was received. egory has the following elements: The Language element is described above and identifies the language in which the plot is described in the Value 60 element. Value MSI Content Note Entry Index Date Time The Content Note metadata category is used to store any Entry Tag other text associated with the media content. The structure and type of text is deliberately not specified in order to 65 The Value element stores an indication of the duration of provide extensibility. An example entity within the Content the media content. For example, a two-hour movie has a Note metadata category has the following elements: duration of two hours, or 120 minutes. US 7,363,649 B2 39 40 The Entry Index and Entry Tag elements are described Popularity above and indicate that a metadata provider may submit The Popularity metadata category is used to store data that multiple opinions as to the duration of the media content, or describes how popular the media content is. An example a composite description may indicate various opinions as to entity within the Popularity metadata category has the the duration, the opinions Submitted by multiple metadata following elements: providers. For example, metadata providers may have dif fering opinions about the duration of the content based on whether or not the preamble at the front end of a movie is counted. Popularity Country Entry Index MSI Rating Year Entry Tag Date Time The MSI element is described above and identifies the 10 metadata provider. The Date Time element is described above and indicates The Popularity Rating element stores an indicator of the the date and time at which the data was received. level of popularity of the media content. Example values Popularity 610 may include: Unknown, None, and any integer from 1 to 10. FIG. 11 illustrates example popularity categories. Popu 15 The Country element stores a value that indicates the larity 610 includes metadata categories that indicate how country associated with the popularity rating given in the popular the media content is. Example popularity categories Popularity Rating element. A list of possible values for include critic reviews, popularity, and broadcast ubiquity. populating the Country element is given in Table 9. The Year element stores the year associated with the Critic Reviews popularity rating given in the Popularity Rating element. The Critic Reviews metadata category is used to store The Entry Index and Entry Tag elements are described data associated with a review of the media content by a above and indicate that a metadata provider may submit critic. An example entity within the Critic Reviews metadata more than one opinion as to the popularity of the media category has the following elements: content, or a composite description may indicate various 25 opinions as to the popularity, the opinions Submitted by multiple metadata providers. Star Rating Fourth Axis Rating Entry Index The MSI element is described above and identifies the First Axis Meaning Fourth Axis Meaning Entry Tag metadata provider. Second Axis Rating Fifth Axis Rating MSI The Date Time element is described above and indicates Second Axis Meaning Fifth Axis Meaning Date Time 30 Third Axis Rating Review Text Language the date and time at which the data was received. Third Axis Meaning Reviewer Broadcast Ubiquity The Broadcast Ubiquity metadata category is used to store The Star Rating, Second Axis Rating. Third Axis Rating, a number indicating how often a particular piece of content Fourth Axis Rating, and Fifth Axis Rating elements each 35 is broadcast. Some programs are repeated over and over store an indicator of a rating of the media content, for (e.g., episodes of “Friends' and “The Simpsons'), whereas example according to a rating scale. Example values may Some programs (e.g., a documentary about a particular tribe include: Unknown, None, and any integer from 1 to 10. of Indians) might only get broadcast once. An example The First Axis Meaning, Second Axis Rating. Third Axis entity within the Broadcast Ubiquity metadata category has Rating, Fourth Axis Rating, and Fifth Axis Rating elements 40 the following elements: store indicators of the type of ratings given in the Star Rating, Second Axis Rating. Third Axis Rating, Fourth Axis Rating, and Fifth Axis Rating elements, respectively. For Ubiquity Country Entry Index MSI example, the Star Rating element may have a value of 5 and Rating Year Entry Tag Date Time the Second Axis Rating may have a value of 10. Addition 45 ally, the First Axis Meaning element may have a value of The Ubiquity Rating element stores information about “Overall” and the Second Axis Meaning element may have how often a particular program is broadcast. This informa a value of “Comedy Content”. This indicates that the Star tion is useful to a user when looking through an electronic Rating of 5 is an overall rating and the First Axis Rating of program guide to assess how important it is for the user to 10 is a rating of the comedy within the media content. 50 view a particular instance of a program. A fully automated The Review Text element stores text associated with the Personal Video Recorder system may use this information to review that is generated by a reviewer of the media content. make decisions when resolving record conflicts. Example The Reviewer element stores the name of the media values may include: Unknown, None, and any integer from 1 to 10. content reviewer (i.e., the person who wrote the review). 55 The Entry Index and Entry Tag elements are described The Country element stores a value that indicates the above and indicate that a metadata provider may submit country associated with the ubiquity rating given in the more than one review of the media content, or a composite Ubiquity Rating element. A list of possible values for description may indicate various reviews Submitted by mul populating the Country element is given in Table 9. tiple metadata providers. 60 The Year element stores the year associated with the The MSI element is described above and identifies the ubiquity rating given in the Ubiquity Rating element. metadata provider. The Entry Index and Entry Tag elements are described The Date Time element is described above and indicates above and indicate that a metadata provider may submit the date and time at which the data was received. more than one opinion as to the ubiquity rating of the media The Language element is described above and identifies 65 content, or a composite description may indicate various the language in which the text in the Review Text element opinions as to the broadcast ubiquity, the opinions Submitted is written. by multiple metadata providers. US 7,363,649 B2 41 42 The MSI element is described above and identifies the which the media content contains adult content. An example metadata provider. entity within the Extended Censor Rating Adult Content The Date Time element is described above and indicates metadata category has the following elements: the date and time at which the data was received. Censor Ratings 612 FIG. 12 illustrates example censor ratings categories. Value MSI Censor ratings 612 includes metadata categories that indi Entry Index Date Time cate censorship ratings associated with the media content. Entry Tag Example censor ratings categories include censor parental rating, extended censor rating adult content, extended censor 10 rating bad language, extended censor rating violence, The Value element stores an indicator of the degree of extended censor rating nudity, and extended censor rating adult content found in the media content. Example indica drug abuse. tors may include: Unknown, None, Brief, Significant, and Censor Parental Rating Extreme. 15 The Entry Index and Entry Tag elements are described The Censor Parental Rating metadata category is used to above and indicate that a metadata provider may submit store data identifying whether the media contains content more than one opinion as to the degree of adult content that may be inappropriate for Some individuals, such as found in the media content, or a composite description may children under a particular age. An example entity within the indicate various opinions as to the degree of adult content, Censor Parental Rating metadata category has the following the opinions submitted by multiple metadata providers. For elements: example, different extended censor ratings may be applied to the same media content across different countries. The MSI element is described above and identifies the Censor Category Entry Index MSI Language metadata provider. Entry Tag Date Time Advisory Comment 25 The Date Time element is described above and indicates the date and time at which the data was received. The Censor Category element stores an indicator of Extended Censor Rating Bad Language whether the media contains content that may be inappropri The Extended Censor Rating Bad Language metadata ate for some individuals. Example values for the Censor category is used to store data that indicates the degree to Category element are given in Table 13. 30 which the media content contains bad language. An example entity within the Extended Censor Rating Bad Language TABLE 13 metadata category has the following elements: Unknown None 35 Suitable For Everyone (U.G.TVY) Value MSI Unsuitable For Children Under 10 (PG.TVY7,TVPG) Entry Index Date Time Unsuitable For Children Under 13 (PG13,TV14) Entry Tag Unsuitable For Children Under 17 (R) Unsuitable For Many Adults (TVMA, 18XXXX) Not Rated The Value element stores an indicator of the degree of bad 40 language found in the media content. Example indicators The Advisory Comment element stores text that describes may include: Unknown, None, Brief, Significant, and attributes of the media content that may be objectionable or Extreme. inappropriate for some individuals. For example, when a The Entry Index and Entry Tag elements are described rating authority rates a movie in a particular category, they 45 above and indicate that a metadata provider may submit may also indicate the reason for their decision, for example, more than one opinion as to the degree of bad language excessive violence or bad language. In an increasing trend, found in the media content, or a composite description may this information is now also appearing in radio and televi indicate various opinions as to the degree of bad language, sion advertisments for movies. the opinions submitted by multiple metadata providers. For The Entry Index and Entry Tag elements are described 50 example, different extended censor ratings may be applied to above and indicate that a metadata provider may submit the same media content across different countries. more than one censor parental rating of the media content, The MSI element is described above and identifies the or a composite description may indicate various opinions as metadata provider. to the censor parental rating, the opinions Submitted by The Date Time element is described above and indicates multiple metadata providers. For example, different censor 55 the date and time at which the data was received. ratings may be applied to the same media content across Extended Censor Rating Violence different countries. The Extended Censor Rating Violence metadata category The MSI element is described above and identifies the is used to store data that indicates the degree to which the metadata provider. media content contains violence. An example entity within The Date Time element is described above and indicates 60 the Extended Censor Rating Violence metadata category has the date and time at which the data was received. the following elements: The Language element is described above and identifies the language in which the text in the Advisory Comment element is written. Value MSI Entry Index Date Time Extended Censor Rating Adult Content 65 Entry Tag The Extended Censor Rating Adult Content metadata category is used to store data that indicates the degree to US 7,363,649 B2 43 44 The Value element stores an indicator of the degree of example, different extended censor ratings may be applied to violence found in the media content. Example indicators the same media content across different countries. may include: Unknown, None, Brief, Significant, and The MSI element is described above and identifies the Extreme. metadata provider. The Entry Index and Entry Tag elements are described 5 The Date Time element is described above and indicates above and indicate that a metadata provider may submit the date and time at which the data was received. more than one opinion as to the degree of violence found in Technical Details 614 the media content, or a composite description may indicate FIG. 13 illustrates example technical details categories. various opinions as to the degree of violence, the opinions Technical details 614 includes metadata categories that submitted by multiple metadata providers. For example, 10 indicate technical details associated with the media content different extended censor ratings may be applied to the same or the production or capture of the media content. Example media content across different countries. technical details categories include capture mechanism, The MSI element is described above and identifies the image capture resolution, video capture temporal rate HZ, metadata provider. Video captured using interlace, Sound capture sampling, The Date Time element is described above and indicates 15 Sound capture compression, camera used, image capture the date and time at which the data was received. compression, recorded live, black and white, silent, post Extended Censor Rating Nudity production processing, special electronic processing, aspect The Extended Censor Rating Nudity metadata category is ratio, aspect ratio implementation, pan and Scan vectors, used to store data that indicates the degree to which the origination note, Stereo Sound, and Surround Sound. media content contains nudity. An example entity within the Capture Mechanism Extended Censor Rating Nudity metadata category has the The Capture Mechanism metadata category is used to following elements: store data that identifies how the media content was cap tured. An example entity within the Capture Mechanism 25 metadata category has the following elements: Value MSI Entry Index Date Time Entry Tag Value MSI Entry Index Date Time The Value element stores an indicator of the degree of 30 Entry Tag nudity found in the media content. Example indicators may include: Unknown, None, Brief, Significant, and Extreme. The Value element stores an indicator of the mechanism The Entry Index and Entry Tag elements are described used to capture the media content. Example capture mecha above and indicate that a metadata provider may submit nism indicators are given in Table 14. more than one opinion as to the degree of nudity found in the 35 media content, or a composite description may indicate various opinions as to the degree of nudity, the opinions TABLE 1.4 submitted by multiple metadata providers. For example, Unknown 16mm Film Digital Audio different extended censor ratings may be applied to the same Film Bigger Than 35mm Video Camera Other media content across different countries. 40 35mm Film Analog Audio The MSI element is described above and identifies the metadata provider. The Entry Index and Entry Tag elements are described The Date Time element is described above and indicates above and indicate that a metadata provider may submit the date and time at which the data was received. more than one opinion as to the mechanism used to capture Extended Censor Rating Drug Abuse 45 the media content. This may indicate that portions of the The Extended Censor Rating Drug Abuse metadata cat media content were captured using different capture mecha egory is used to store data that indicates the degree to which nisms. In a composite description the Entry Index and Entry the media content portrays drug abuse. An example entity Tag elements may indicate various opinions as to the capture within the Extended Censor Rating Drug Abuse metadata mechanism, the opinions Submitted by multiple metadata category has the following elements: 50 providers The MSI element is described above and identifies the metadata provider. Value MSI The Date Time element is described above and indicates Entry Index Date Time the date and time at which the data was received. Entry Tag 55 Image Capture Resolution The Image Capture Resolution metadata category is used The Value element stores an indicator of the degree of to store data that indicates the resolution of the originally drug abuse portrayed in the media content. Example indi captured media content image. An example entity within the cators may include: Unknown, None, Brief, Significant, and 60 Image Capture Resolution metadata category has the fol Extreme. lowing elements: The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit more than one opinion as to the degree of drug abuse found Horizontal Entry Index MSI in the media content, or a composite description may indi 65 Vertical Entry Tag Date Time cate various opinions as to the degree of drug abuse, the opinions submitted by multiple metadata providers. For US 7,363,649 B2 45 The Horizontal element stores a positive integer that represents the horizontal resolution of the originally cap tured media content image. Value MSI The Vertical element stores a positive integer that repre Entry Index Date Time sents the vertical resolution of the originally captured media Entry Tag content image. The Entry Index and Entry Tag elements are described The Value element stores an indicator of whether or not above and indicate that a metadata provider may submit the media content was captured using interlace. Example more than one opinion as to the capture resolution of the 10 values may include: Unknown, True, and False. media content, or a composite description may indicate The Entry Index and Entry Tag elements are described various opinions as to the capture resolution, the opinions above and indicate that a metadata provider may submit submitted by multiple metadata providers. For example, if more than one opinion as to whether the media content was Some areas of a camera's sensor are marked off, metadata captured using interlace, or a composite description may providers may have different opinions of what capture 15 indicate various opinions as to whether or not the media content was captured using interlace, the opinions Submitted resolution is associated with the media content. by multiple metadata providers. This allows for differing The MSI element is described above and identifies the opinions among metadata providers as to whether or not the metadata provider. media content was captured using interlace. The Date Time element is described above and indicates The MSI element is described above and identifies the the date and time at which the data was received. metadata provider. Video Capture Temporal Rate HZ The Date Time element is described above and indicates The Video Capture Temporal Rate Hz metadata category the date and time at which the data was received. is used to store data that indicates how often the original Sound Capture Sampling scene was captured by the camera. For example, a film 25 The Sound Capture Sampling metadata category is used to store data that indicates technical details describing how camera runs at a rate of 24 captures per second, whereas a Sound was captured in association with the media content. Video camera captures at a rate of 60 samples per second. An An example entity within the Sound Capture Sampling example entity within the Video Capture Temporal Rate HZ metadata category has the following elements: metadata category has the following elements: 30

Sample Rate KHZ Entry Index Date Time Value MSI Bits Per Sample Entry Tag Entry Index Date Time Number Of Channels MSI Entry Tag 35 The Sample Rate KHZ element stores the frequency at The Value element stores the frequency at which the which the analog audio waveform was sampled to make the camera captured the scene, such as how many times a digital representation. second it took a picture of the actors to form the video signal. The Bits Per Sample element specifies the number of bits The Entry Index and Entry Tag elements are described above 40 used in the analog to digital converter used to convert the and indicate that a metadata provider may Submit more than analog waveform into the digital representation. one opinion as to the temporal rate at which the video was The Number Of Channels element specifies the number of captured, or a composite description may indicate various audio channels that were captured. Stereo Sound is 2 chan opinions as to the temporal rate, the opinions Submitted by 45 nels whereas Surround sound is typically 6 channels or more. multiple metadata providers. The Entry Index and Entry Tag elements are described The MSI element is described above and identifies the above and indicate that a metadata provider may submit metadata provider. more than one opinion relating to Sound capture sampling, or a composite description may indicate various opinions The Date Time element is described above and indicates relating to Sound capture sampling, the opinions Submitted the date and time at which the data was received. 50 by multiple metadata providers. Video Captured Using Interlace The MSI element is described above and identifies the The Video Captured Using Interlace metadata category is metadata provider. used to store data that indicates whether the media content The Date Time element is described above and indicates was captured using interlace. Interlace is a scanning tech 55 the date and time at which the data was received. nique used by Some cameras in which the camera only Sound Capture Compression captures half the resolution of the scene at each temporal The Sound Capture Compression metadata category is sample point, but does it at a temporal rate that is typically used to store data that indicates the type of compression twice what it could handle if it was capturing the full utilized when capturing the Sound associated with the media resolution at each temporal sample point. Capturing with 60 content. An example entity within the Sound Capture Com interlace has many negative implications through the video pression metadata category has the following elements: delivery chain. Information about whether the original scene was captured using interlace or whether it was changed to Type Entry Tag interlace later is helpful in reducing the negative effects of Ratio MSI interlace. An example entity within the Video Captured 65 Entry Index Date Time Using Interlace metadata category has the following ele ments: US 7,363,649 B2 47 48 The Type element stores an indicator of the type of The Ratio element stores a number that indicates the level compression used. Example values may include: Unknown, of compression. None, MP3, WindowsMedia, Real, and Other. The Entry Index and Entry Tag elements are described The Ratio element stores a number that indicates the level above and indicate that a metadata provider may submit of compression. more than one opinion as to the image compression type The Entry Index and Entry Tag elements are described associated with the capture of the media content, or a above and indicate that a metadata provider may submit composite description may indicate various opinions as to more than one opinion as to the sound compression type the image compression type, the opinions Submitted by associated with the capture of the media content, or a multiple metadata providers. composite description may indicate various opinions as to 10 The MSI element is described above and identifies the the Sound compression type, the opinions Submitted by metadata provider. multiple metadata providers. The Date Time element is described above and indicates The MSI element is described above and identifies the the date and time at which the data was received. metadata provider. Recorded Live The Date Time element is described above and indicates 15 The Recorded Live metadata category is used to store an the date and time at which the data was received. indicator of whether or not the media content was recorded Camera Used live. Media content that was recoded live may include a The Camera Used metadata category is used to store data music album that consists of a recording generated at a that indicates the type of camera used to capture the media concert. An example entity within the Recorded Live meta content. An example entity within the Camera Used meta data category has the following elements: data category has the following elements:

Value MSI Entry Index Date Time Value MSI 25 Entry Tag Entry Index Date Time Entry Tag The Value element stores an indicator of whether or not The Value element stores a name or description of the the media content was recorded live. Example values may camera used to capture the media content. 30 include: Unknown, True, and False. The Entry Index and Entry Tag elements are described The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit data above and indicate that a metadata provider may submit that indicates that more than one type of camera was used to more than one opinion as to whether the media content was capture the media content or the metadata provider may recorded live, or a composite description may indicate Submit multiple opinions as to the type of camera used. In 35 various opinions as to the whether the media content was a composite description, the Entry Index and Entry Tag recorded live, the opinions submitted by multiple metadata elements may differentiate between various opinions as to providers. the camera used, the opinions Submitted by multiple meta The MSI element is described above and identifies the data providers. metadata provider. The MSI element is described above and identifies the 40 The Date Time element is described above and indicates metadata provider. the date and time at which the data was received. The Date Time element is described above and indicates Black And White the date and time at which the data was received. The Black And White metadata category is used to store Image Capture Compression an indicator of whether or not the media content is black and 45 white. An example entity within the Black And White The Image Capture Compression metadata category is metadata category has the following elements: used to store data that indicates how one or more images associated with the media content were compressed upon capture. An example entity within the Image Capture Com pression metadata category has the following elements: 50 Value MSI Entry Index Date Time Entry Tag

Type Entry Tag Ratio MSI The Value element stores an indicator of whether or not Entry Index Date Time 55 the media content is black and white. Example values may include: Unknown, True, and False. The Entry Index and Entry Tag elements are described The Type element stores an indicator of the type of image above and indicate that a metadata provider may submit compression used. Example types of image compression are more than one opinion as to whether the media content is given in Table 15. 60 black and white, or a composite description may indicate various opinions as to whether the media content is black TABLE 1.5 and white, the opinions submitted by multiple metadata Unknown JPEG2000 MPEG2 Real providers. None DV MPEG4 Other The MSI element is described above and identifies the JPEG MPEG1 WindowsMedia 65 metadata provider. The Date Time element is described above and indicates the date and time at which the data was received. US 7,363,649 B2 49 Silent The Silent metadata category is used to store an indicator of whether the media content is silent, Such as a silent movie. Value MSI An example entity within the Silent metadata category has Entry Index Date Time the following elements: 5 Entry Tag The Value element stores an indicator of a type of special Value MSI electronic processing that was performed on the media Entry Index Date Time 10 content. Example types of special electronic processing are Entry Tag listed in Table 17.

The Value element stores an indicator of whether or not TABLE 17 the media content is silent. Example values may include: Unknown Noise Filtered Unknown, True, and False. 15 None Upconverted Digitally Remastered Other The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit more than one opinion as to whether the media content is The Entry Index and Entry Tag elements are described silent, or a composite description may indicate various above and indicate that a metadata provider may submit opinions as to whether the media content is silent, the more than one indicator of special electronic processing opinions Submitted by multiple metadata providers. performed on the media content, or a composite description The MSI element is described above and identifies the may indicate various opinions as to the special electronic metadata provider. processing performed, the opinions Submitted by multiple 25 metadata providers. The Date Time element is described above and indicates The MSI element is described above and identifies the the date and time at which the data was received. metadata provider. Post Production Processing The Date Time element is described above and indicates The Post Production Processing metadata category is used the date and time at which the data was received. to store data that indicates the degree to which post produc 30 Aspect Ratio tion processing was performed on the media content. An example entity within the Post Production Processing meta The Aspect Ratio metadata category is used to store data data category has the following elements: that indicates the ratio of the height compared with the width of the video. Content made for television is typically 3 units high for every 4 units wide, whereas movies typically have 35 a much wider aspect ratio. An example entity within the Value MSI Aspect Ratio metadata category has the following elements: Entry Index Date Time Entry Tag

40 Value MSI The Value element stores an indicator of the degree to Entry Index Date Time which post production processing was performed on the Entry Tag media content. Example values are given in Table 16. The Value element stores the ratio of the width of the TABLE 16 45 Video content (measured in pixels) compared to the height of Unknown Some Post Production the video content (also measured in pixels). Live Extensive Post Production The Entry Index and Entry Tag elements are described Recorded Live Other above and indicate that a metadata provider may submit more than one opinion of what the aspect ratio of the media The Entry Index and Entry Tag elements are described 50 content is, or a composite description may indicate various above and indicate that a metadata provider may submit opinions as to the aspect ratio, the opinions Submitted by more than one opinion as to the degree of post production multiple metadata providers. processing performed on the media content, or a composite The MSI element is described above and identifies the description may indicate various opinions as to the degree of metadata provider. 55 post production processing, the opinions Submitted by mul The Date Time element is described above and indicates tiple metadata providers. the date and time at which the data was received. The MSI element is described above and identifies the Aspect Ratio Implementation metadata provider. The Aspect Ratio Implementation metadata category is The Date Time element is described above and indicates 60 used to store data that indicates how the actual aspect ratio the date and time at which the data was received. was conveyed in the video transport mechanism. Sometimes Special Electronic Processing it is necessary to compress the number of pixels in a The Special Electronic Processing metadata category is particular dimension in order for the media content to fit used to store data that indicates a type of special electronic within the rigid specifications of a transport mechanism, processing that was performed on the media content. An 65 such as a television broadcast system or a DVD. An example example entity within the Special Electronic Processing entity within the Aspect Ratio Implementation metadata metadata category has the following elements: category has the following elements: US 7,363,649 B2 52 unstructured to allow for extensibility. An example entity within the Origination Note metadata category has the Value MSI following elements: Entry Index Date Time Entry Tag Value Entry Tag Date Time The Value element stores an indicator of the aspect ratio Entry Index MSI Language implementation used. Example values are given in Table 18. 10 The Value element stores any additional information TABLE 1.8 about the media origination process. Unknown Letterboxed In 1.78 None Anamorphically Squeezed The Entry Index and Entry Tag elements are described Native Horizontally Oversampled above and indicate that a metadata provider may submit Letterboxed In 1.33 Other more than one origination note associated with the media 15 content, or a composite description may indicate various The Entry Index and Entry Tag elements are described origination notes Submitted by multiple metadata providers. above and indicate that a metadata provider may submit The MSI element is described above and identifies the more than one opinion of what aspect ratio implementation metadata provider. was used, or a composite description may indicate various The Date Time element is described above and indicates opinions as to the aspect ratio implementation, the opinions the date and time at which the data was received. submitted by multiple metadata providers. The Language element is described above and identifies The MSI element is described above and identifies the the language in which the text in the Value element is metadata provider. written. 25 The Date Time element is described above and indicates Stereo Sound the date and time at which the data was received. Pan and Scan Vectors The Stereo Sound metadata category is used to store data indicating whether the media content includes stereo sound. The Pan And Scan Vectors metadata category is used to An example entity within the Stereo Sound metadata cat store data that indicates whether the media content has pan 30 and scan vectors available with the content. Pan and scan egory has the following elements: vectors provide a mechanism for selecting a 4:3 aspect ratio image from a much wider film image. For example, a movie with a wide aspect ratio does not fit the aspect ratio available Value MSI on a standard television. In a simple case if the action is 35 Entry Index Date Time more on the left of the film frame then the vectors would Entry Tag indicate that the 4:3 image is to be extracted from the left of the film frame. As the action appears more on the right side of the film frame, the vectors would indicate that the 4:3 The Value element stores an indicator of whether or not television frame should be extracted from the right of the the media content includes stereo Sound. Example values film frame. An example entity within the Pan And Scan 40 may include: Unknown, True, and False. Vectors metadata category has the following elements: The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit more than one opinion as to whether the media content includes stereo Sound, or a composite description may Value MSI 45 Entry Index Date Time indicate various opinions as to the inclusion of stereo sound, Entry Tag the opinions submitted by multiple metadata providers. The MSI element is described above and identifies the metadata provider. The Value element stores an indicator of whether or not the media content has pan and scan vectors available. 50 The Date Time element is described above and indicates Example values may include: Unknown, True, and False. the date and time at which the data was received. The Entry Index and Entry Tag elements are described Surround Sound above and indicate that a metadata provider may submit The Surround Sound metadata category is used to store more than one opinion as to whether the media content has data indicating whether the media content is encoded to pan and scan vectors available, or a composite description 55 Support Surround Sound. An example entity within the Sur may indicate various opinions as to the availability of pan round Sound metadata category has the following elements: and scan vectors, the opinions Submitted by multiple meta data providers. The MSI element is described above and identifies the 60 Encoding Method Entry Index Date Time metadata provider. Number Of Channels Entry Tag Language The Date Time element is described above and indicates Mix Notes MSI the date and time at which the data was received. Origination Note The Encoding Method element stores a value that indi The Origination Note metadata category is used to store 65 cates the method used to encode the media content to any additional information about the technical details of the Support Surround Sound. Example indicators of Surround media origination process. This category is deliberately Sound encoding methods are given in Table 19. US 7,363,649 B2 53 54 Budget TABLE 19 The Budget metadata category is used to store data that Unknown AC-3 - Dolby Digital AC-3 indicates the media content creation budget. An example None DTS - Digital Theatre Sound entity within the Budget metadata category has the follow Dolby Surround SDDS - Sony Dynamic Digital Sound ing elements: Dolby Pro- MLP - Meridian Lossless Packing Logic Dolby Pro- Other Logic with THX Value Entry Tag Date Time Entry Index MSI 10 The Number Of Channels element specifies the number of channels used to create the surround sound effect. Some The Value element stores an indicator of the budget used systems use 6 channels whereas others use 8 channels. in creating the media content. Example values may include: The Mix Notes element specifies the technical details as Unknown, Amateur, Low, Medium, and High. to how the surround sound effect was mixed together to 15 The Entry Index and Entry Tag elements are described achieve the desired result for the listener. above and indicate that a metadata provider may submit data The Entry Index and Entry Tag elements are described indicating that more than one opinion as to the budget above and indicate that a metadata provider may submit data associated with the media content, or a composite descrip indicating that more than one surround sound encoding tion may indicate various opinions as to the budget, the method was used in association with the media content, or opinions Submitted by multiple metadata providers. a composite description may indicate various opinions as to The MSI element is described above and identifies the the Surround sound encoding method, the opinions Submit metadata provider. ted by multiple metadata providers. The Date Time element is described above and indicates The MSI element is described above and identifies the the date and time at which the data was received. metadata provider. 25 Box Office The Date Time element is described above and indicates The Box Office metadata category is used to store data the date and time at which the data was received. indicating the box office income generated by the media The Language element is described above and identifies content. An example entity within the Box Office metadata the language in which the text in the Mix Notes element is 30 category has the following elements: written. Production Details 616 FIG. 14 illustrates example production details categories. USAOpeningWeekendCinemaMillionDollars Entry Index Production details 616 includes metadata categories related USATotalGrossCinemaMillionDollars Entry Tag 35 Worldwide.TotalGrossCinemaMillionDollars MSI to the production of the media content. Example production Comment Date Time details categories include made for, budget, box office, production company, distribution company, principal coun try of main production company, capture location, copy The USAOpeningWeekendCinemaMillion Dollars ele right, and URL production company. ment stores an integer number indicating how many millions Made For 40 of dollars the media content generated during the first weekend it was available in theaters in the United States. The Made For metadata category is used to store data that The USATotalGrossCinemaMillionDollars element stores identifies the purpose for which the media content was an integer number indicating how many millions of dollars created. An example entity within the Made For metadata the media content generated while it was available in category has the following elements: 45 theaters in the United States. The WorldwideTotalGrossCinemaMillionDollars element stores an integer number indicating how many millions of Value MSI dollars the media content generated while it was available in Entry Index Date Time theaters throughout the world. Entry Tag 50 The Comment element may be used to store any addi tional details as to how the figure was arrived at. For The Value element stores an indicator of the purpose for example, the figure may only included specific geographic which the media content was created. Example values may aaS. include: Unknown, TV, Cinema, Internet, Amateur Use, The Entry Index and Entry Tag elements are described Military, and Other. 55 above and indicate that a metadata provider may submit The Entry Index and Entry Tag elements are described multiple opinions as to the box office monetary Success of above and indicate that a metadata provider may submit data the media content, or a composite description may indicate indicating more than one purpose for creation associated various opinions as to the box office Success, the opinions with the media content, or a composite description may submitted by multiple metadata providers. indicate various opinions as to who or what the media 60 The MSI element is described above and identifies the content was made for, the opinions Submitted by multiple metadata provider. metadata providers. The Date Time element is described above and indicates The MSI element is described above and identifies the the date and time at which the data was received. metadata provider. 65 Production Company The Date Time element is described above and indicates The Production Company metadata category is used to the date and time at which the data was received. store data that identifies a production company that pro US 7,363,649 B2 55 56 duced the media content. An example entity within the The Value element stores a value that indicates the coun Production Company metadata category has the following try with which the main production company is associated. elements: A list of possible values for indicating the country is given in Table 9.

Value MSI The Entry Index and Entry Tag elements are described Entry Index Date Time above and indicate that a metadata provider may submit data Entry Tag indicating that the main production company is associated with more than one country, or a composite description may The Value element stores the name of a production 10 indicate various opinions as to the country associated with company associated with the media content. the main production company, the opinions Submitted by The Entry Index and Entry Tag elements are described multiple metadata providers. above and indicate that a metadata provider may submit data The MSI element is described above and identifies the identifying a plurality of production companies associated metadata provider. with the media content, or a composite description may 15 indicate various opinions as to the production company, the The Date Time element is described above and indicates opinions Submitted by multiple metadata providers. the date and time at which the data was received. The MSI element is described above and identifies the Capture Location metadata provider. The Capture Location metadata category is used to store The Date Time element is described above and indicates data identifying where the media content was captured. An the date and time at which the data was received. example entity within the Capture Location metadata cat Distribution Company egory has the following elements: The Distribution Company metadata category is used to store data that identifies a distribution company that is 25 associated with the media content. An example entity within Country Capture Location Detail MSI the Distribution Company metadata category has the fol State Entry Index Date Time lowing elements: City Entry Tag Language

Company Name Entry Tag 30 The Country element stores a value that indicates the Geographic Area MSI Distribution Medium Date Time country in which the media content was captured. A list of Entry Index possible values for populating the Country element is given in Table 9. The Company Name element stores the name of a distri The State element stores a value that indicates the state in bution company associated with the media content. 35 which the media content was captured. The value may be the The Geographic Area element stores text that identifies name of a state, such as “Washington', or an abbreviation the geographic area that the particular distribution company associated with the state, such as “WA'. is responsible for. The City element stores the name of the city in which the The Distribution Medium element stores text that identi 40 media content was captured. fies the medium through which the distribution company The Capture Location Detail element is used to store distributes the media content. For example, distribution additional details that describe the location where the media mediums may include: CD, DVD, video cassette, broadcast content was captured. If the movie was filmed on location, rights, pay per view, and internet streaming rights. the Capture Location Detail element may contain the value, The Entry Index and Entry Tag elements are described 45 above and indicate that a metadata provider may submit data “Alcatraz Island in the middle of San Francisco Bay,” in a indicating more than one distribution company associated Capture Location element associated with the movie, with the media content, or a composite description may “Escape from Alcatraz’. indicate various opinions as to the distribution company, the The Entry Index and Entry Tag elements are described opinions Submitted by multiple metadata providers. 50 above and indicate that a metadata provider may submit data The MSI element is described above and identifies the indicating more than one location at which the media metadata provider. content was captured, or a composite description may indi The Date Time element is described above and indicates cate various opinions as to the capture location, the opinions the date and time at which the data was received. submitted by multiple metadata providers. Principal Country of Main Production Company 55 The MSI element is described above and identifies the The Principal Country Of Main Production Company metadata provider. metadata category is used to store data identifying the The Date Time element is described above and indicates principal country with which the main production company associates. An example entity within the Principal Country the date and time at which the data was received. Of Main Production Company metadata category has the 60 The Language element is described above and identifies following elements: the language in which the text in the Capture Location Detail element is written. Copyright Value MSI Entry Index Date Time The Copyright metadata category is used to store data Entry Tag 65 indicating copyright information associated with the media content. An example entity within the Copyright metadata category has the following elements: US 7,363,649 B2 58 actress is wearing. An example entity within the ATVEF Data Enhancement metadata category has the following Owner Entry Index MSI elements: Year Entry Tag Date Time

The Owner element stores the name of the copyright Value MSI OW. Entry Index Date Time The Year element stores the number representing the year Entry Tag associated with the copyright. 10 The Entry Index and Entry Tag elements are described The Value element stores an indicator of whether or not above and indicate that a metadata provider may submit data ATVEF data enhancement has been applied to the media indicating more than one copyright associated with the content. Example values may include: Unknown, True, and media content, or a composite description may indicate False. various opinions as to the copyright details, the opinions 15 submitted by multiple metadata providers. The Entry Index and Entry Tag elements are described The MSI element is described above and identifies the above and indicate that a metadata provider may submit metadata provider. more than one opinion as to whether or not ATVEF data The Date Time element is described above and indicates enhancement has been applied to the media content, or a the date and time at which the data was received. composite description may indicate various opinions as to URL Production Company whether or not ATVEF data enhancement has been applied, The URL Production Company metadata category is used the opinions submitted by multiple metadata providers. to store the address of a website associated with a production company that is associated with the media content. An The MSI element is described above and identifies the example entity within the URL Production Company meta 25 metadata provider. data category has the following elements: The Date Time element is described above and indicates the date and time at which the data was received. Educational Commands Value MSI The Educational Commands metadata category is used to Entry Index Date Time 30 store data indicating whether educational commands have Entry Tag been added to the media content. Educational commands are implemented similarly to ATVEF enhancements and are The Value element stores the production company's web used to add educational information to video content. An site address. 35 example entity within the Educational Commands metadata The Entry Index and Entry Tag elements are described category has the following elements: above and indicate that a metadata provider may submit data identifying more than one website associated with a pro duction company that is associated with the media content, Value Entry Tag Date Time or a composite description may indicate various opinions as 40 to the URL, the opinions submitted by multiple metadata Entry Index MSI providers. The MSI element is described above and identifies the The Value element stores an indicator of whether educa metadata provider. tional commands have been added to the media content. The Date Time element is described above and indicates 45 Example values may include: Unknown, True, and False. the date and time at which the data was received. The Entry Index and Entry Tag elements are described Enhancements 618 above and indicate that a metadata provider may submit FIG. 15 illustrates example enhancements categories. more than one opinion as to whether educational commands Enhancements 618 includes metadata categories describing have been added to the media content, or a composite post-production enhancements to the media content. 50 Example enhancements metadata categories include ATVEF description may indicate various opinions as to whether data enhancement, educational commands, educational educational commands have been added, the opinions Sub informational enhancements, multiple camera angles, mul mitted by multiple metadata providers. tiple story lines, phone in, URL enhancements, URL more The MSI element is described above and identifies the info, associated phone number, associated teletext page 55 metadata provider. number, and VCR programming code. The Date Time element is described above and indicates ATVEF Data Enhancement the date and time at which the data was received. The ATVEF Data Enhancement metadata category is used Educational Informational Enhancements to store data indicating whether ATVEF data enhancement The Educational Informational Enhancements metadata has been applied to the media content. ATVEF is an acroym 60 for the Advanced Television Enhancement Forum. The category is used to store data indicating whether educational ATVEF enhancement mechanism allows video content to be informational enhancements have been added to the media annotated with extra information. For example, an interac content. As with ATVEF enhancements and educational tive button may be embedded, that when selected, causes commands, educational informational enhancements are additional information about an actress to be displayed on 65 textual enhancements to video content. An example entity the screen. Another enhancement may provide a mechanism within the Educational Informational Enhancements meta that allows a viewer to purchase the same dress that an data category has the following elements: US 7,363,649 B2 60 indicate various opinions regarding multiple story lines, the opinions Submitted by multiple metadata providers. The MSI element is described above and identifies the Value Entry Tag Date Time Entry Index MSI metadata provider. The Date Time element is described above and indicates the date and time at which the data was received. The Value element stores an indicator of whether educa Phone In tional informational enhancements have been added to the media content. Example values may include: Unknown, The Phone In metadata category is used to store data that indicates whether the audience is invited to phone in to the True, and False. 10 TV station to participate in the program. Viewers may be The Entry Index and Entry Tag elements are described able to phone in to place a vote or they may be invited to above and indicate that a metadata provider may submit give their opinion about something, live on air. An example more than one opinion as to whether educational informa entity within the Phone In metadata category has the fol tional enhancements have been added to the media content, lowing elements: or a composite description may indicate various opinions as 15 to whether educational informational enhancements have been added, the opinions submitted by multiple metadata providers. Value MSI The MSI element is described above and identifies the Entry Index Date Time metadata provider. Entry Tag The Date Time element is described above and indicates the date and time at which the data was received. The Value element stores an indicator of whether the Multiple Camera Angles audience is invited to phone in to the TV station during the The Multiple Camera Angles metadata category is used to program. Example values may include: Unknown, True, and store data indicating whether the media content includes 25 False. multiple camera angles. An example entity within the Mul The Entry Index and Entry Tag elements are described tiple Camera Angles metadata category has the following above and provide a mechanism for a metadata provider to elements: Submit multiple opinions as to the whether or not audience phone in is Supported, or a composite description may 30 indicate various opinions as to phone in Support, the opin Value MSI ions submitted by multiple metadata providers. Entry Index Date Time The MSI element is described above and identifies the Entry Tag metadata provider. The Date Time element is described above and indicates The Value element stores an indicator of whether the 35 the date and time at which the data was received. media content includes multiple camera angles. Example URL Enhancements values may include: Unknown, True, and False. The URL Enhancements metadata category is used to The Entry Index and Entry Tag elements are described store web links assoicated with the media content. If the user above and indicate that a metadata provider may submit goes to the indicated web site then they will find information more than one opinion as to whether the media content 40 that is relevant to the piece of media content, such as a contains multiple camera angles, or a composite description television program. An example entity within the URL may indicate various opinions regarding multiple camera Enhancements metadata category has the following ele angles, the opinions Submitted by multiple metadata provid ments: CS. The MSI element is described above and identifies the 45 metadata provider. The Date Time element is described above and indicates Value MSI the date and time at which the data was received. Entry Index Date Time Multiple Story Lines Entry Tag The Multiple Story Lines metadata category is used to 50 store data indicating whether the media content includes The Value element stores URLs that are associated with multiple story lines. An example entity within the Multiple the media content. Story Lines metadata category has the following elements: The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit 55 more than one associated URL, or a composite description Value MSI may indicate various URLs submitted by multiple metadata Entry Index Date Time providers. Entry Tag The MSI element is described above and identifies the 60 metadata provider. The Value element stores an indicator of whether the The Date Time element is described above and indicates media content includes multiple story lines. Example values the date and time at which the data was received. may include: Unknown, True, and False. URL More Info The Entry Index and Entry Tag elements are described The URL More Info metadata category is used to store above and indicate that a metadata provider may submit 65 additional URLs that are associated with the media content. more than one opinion as to whether the media content An example entity within the URL More Info metadata includes multiple story lines, or a composite description may category has the following elements: US 7,363,649 B2 62 The Date Time element is described above and indicates the date and time at which the data was received.

Value MSI VCR Programming Code Entry Index Date Time The VCR Programming Code metadata category is used Entry Tag to store data indicating a code that may be used to program a VCR to record the media content. An example entity within the VCR Programming Code metadata category has The Value element stores an additional associated URL. the following elements: The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit 10 more than one additional associated URL, or a composite Value MSI description may indicate various URLs submitted by mul Entry Index Date Time tiple metadata providers. Entry Tag The MSI element is described above and identifies the 15 metadata provider. The Value element stores a VCR programming code The Date Time element is described above and indicates associated with the media content. the date and time at which the data was received. The Entry Index and Entry Tag elements are described Associated Phone Number above and indicate that a metadata provider may submit The Associated Phone Number metadata category is used more than one VCR programming code associated with the to store a phone number associated with the media content. media content, or a composite description may indicate An example entity within the Associated Phone Number various VCR programming codes Submitted by multiple metadata category has the following elements: metadata providers. The MSI element is described above and identifies the 25 metadata provider. The Date Time element is described above and indicates Value MSI the date and time at which the data was received. Entry Index Date Time Entry Tag Language Data 620 FIG. 16 illustrates example language data categories. 30 Language data 620 includes metadata categories indicating The Value element stores a phone number associated with languages associated with the media content. Example lan the media content. For example, a televised telethon may guage data categories include language primary original, have one or more phone numbers that views can call to language segment original, language dub, language audio pledge money in response to the telethon. track, language text burnt in, language text track, language The Entry Index and Entry Tag elements are described 35 description track, and sign language track. above and indicate that a metadata provider may submit Language Primary Original more than one phone number associated with the media The Language Primary Original metadata category is used content, or a composite description may indicate various to store data that indicates the primary language of the media phone numbers submitted by multiple metadata providers. content in its original form. An example entity within the The MSI element is described above and identifies the 40 Language Primary Original metadata category has the fol metadata provider. lowing elements: The Date Time element is described above and indicates the date and time at which the data was received.

Associated Teletext Page Number 45 Value MSI The Associated Teletext Page Number metadata category Entry Index Date Time is used to store the page number of a teletext page associated Entry Tag with the media content. An example entity within the Associated Teletext PageNumber metadata category has the The Value element stores an indicator of the primary following elements: 50 original language associated with the media content. Example values for populating the element are given in Table 1.

Value MSI The Entry Index and Entry Tag elements are described Entry Index Date Time above and indicate that a metadata provider may submit data Entry Tag 55 indicating more than one primary original language associ ated with the media content, or a composite description may indicate various opinions as to the primary original lan The Value element stores an integer that corresponds to a guage, the opinions Submitted by multiple metadata provid teletext page number associated with the media content. CS. The Entry Index and Entry Tag elements are described 60 The MSI element is described above and identifies the above and indicate that a metadata provider may submit metadata provider. more than one teletext page associated with the media The Date Time element is described above and indicates content, or a composite description may indicate various the date and time at which the data was received. teletext page numbers Submitted by multiple metadata pro Language Segment Original viders. 65 The Language Segment Original metadata category is The MSI element is described above and identifies the used to store data that indicates another language that metadata provider. appears in at least a segment of the media content. An

US 7,363,649 B2 65 66 The Value element stores an indicator of the text track language associated with the media content. Example values TABLE 20-continued for populating the element are given in Table 1. French Signing Other The Entry Index and Entry Tag elements are described Japanese Signing above and indicate that a metadata provider may submit data indicating that text tracks in more than one language are associated with the media content, or a composite descrip The Spoken Language element stores an indicator of the tion may indicate various text track languages Submitted by spoken language that is associated with the signing. A sign multiple metadata providers. language technique can be used to convey different spoken The MSI element is described above and identifies the 10 languages in different countries. Example values for popu metadata provider. lating the Spoken Language element are given in Table 1. The Date Time element is described above and indicates The Entry Index and Entry Tag elements are described the date and time at which the data was received. above and indicate that a metadata provider may submit data Language Description Track indicating that more than one sign language track is asso The Language Description Track metadata category is 15 ciated with the media content, or a composite description used to store data that indicates the language of a description may indicate various opinions as to the sign language track track associated with the media content. In a description details, the opinions Submitted by multiple metadata pro track, a narrator describes the visual scenes of media con viders. tent. Description tracks are provided to enhance the viewing The MSI element is described above and identifies the experience for the visually impaired. An example entity metadata provider. within the Language Description Track metadata category The Date Time element is described above and indicates has the following elements: the date and time at which the data was received. Schema Implementation Details 622 FIG. 17 illustrates example schema implementation 25 Value MSI details categories. Schema implementation details 622 Entry Index Date Time includes metadata categories that identify where additional Entry Tag files of data associated with the media content are stored. Example schema implementation details metadata catego The Value element stores an indicator of the description ries include XPath content artwork, XPath video trailer, track language associated with the media content. Example 30 XPath chapter thumbnails, XPath time interval thumbnails, values for populating the element are given in Table 1. XPath script or lyrics, and XPath original storyboard. The Entry Index and Entry Tag elements are described XPath Content Artwork above and indicate that a metadata provider may submit data The XPath Content Artwork metadata category is used to indicating that description tracks in more than one language 35 store file paths that indicate where artwork associated with are associated with the media content, or a composite the media content is stored. An example entity within the description may indicate various opinions as to the descrip XPath Content Artwork metadata category has the following tion track language, the opinions Submitted by multiple elements (additional picture sizes may be supported in metadata providers. alternate implementations): The MSI element is described above and identifies the metadata provider. 40 The Date Time element is described above and indicates the date and time at which the data was received. Size 48x64 Entry Index Size 96x128 Entry Tag Sign Language Track Size 192x256 MSI The Sign Language Track is used to store data identifying 45 Size 720x960 Date Time sign language characteristics associated with the media content. An example entity within the Sign Language Track The Size 48x64 element stores a file path that identifies metadata category has the following elements: the storage location of artwork associated with the media content that is 48x64 pixels in resolution. 50 The Size 96x128 element stores a file path that identifies Signing Method Entry Tag the storage location of artwork associated with the media Spoken Language MSI Entry Index Date Time content that is 96x128 pixels in resolution. The Size 192x256 element stores a file path that iden tifies the storage location of artwork associated with the The Signing Method element stores an indicator of the 55 media content that is 192x256 pixels in resolution. sign language method that is associated with the media The Size 720x960 element stores a file path that iden content. Example values for indicating the sign language tifies the storage location of artwork associated with the method are given in Table 20. media content that is 720x960 pixels in resolution. 60 The Entry Index and Entry Tag elements are described TABLE 20 above and indicate that a metadata provider may submit data Unknown Chinese Signing indicating a plurality of artwork associated with the media None American Indian Signing content, or a composite description may indicate various ASL American Sign Language Spanish Signing PSE Pidgin Signed English Slovak Signing opinions as to paths associated with artwork, the opinions SEE Signing Exact English German Signing 65 submitted by multiple metadata providers. Australian Signing Brazilian Signing The MSI element is described above and identifies the metadata provider. US 7,363,649 B2 67 68 The Date Time element is described above and indicates The Date Time element is described above and indicates the date and time at which the data was received. the date and time at which the data was received. XPath Video Trailer XPath Time Interval Thumbnails The XPath Video Trailer metadata category is used to The XPath Time Interval Thumbnails metadata category 5 is used to store a file path that indicates where a set of time store the path to a video trailer associated with the media interval thumbnails associated with the media content is content. An example entity within the XPath Video Trailer stored. Thumbnails are small pictures obtained from video metadata category has the following elements: frame captures. In this case they are captures taken at regular time intervals thus providing a way of navigating to a 10 desired portion of a video without having to scanthrough the File Path Entry Index video itself. An example entity within the XPath Time Resolution Width Entry Tag Interval Thumbnails metadata category has the following Resolution Height MSI elements: Bitrate KBaud Date Time 15

The File Path element stores a file path that identifies the Value MSI storage location of a video trailer associated with the media Entry Index Date Time COntent. Entry Tag The Resolution Width element stores an indicator of the resolution width of the video trailer stored at the location The Value element stores a file path that identifies the indicated by the File Path element. storage location of a set of time interval thumbnails asso The Resolution Height element stores an indicator of the ciated with the media content. resolution height of the video trailer stored at the location The Entry Index and Entry Tag elements are described indicated by the File Path element. 25 above and indicate that a metadata provider may submit data The Bitrate KBaud element stores an indicator of the indicating that more than one set of time interval thumbnails bitrate of the video trailer stored at the location indicated by associated with the media content, or a composite descrip the File Path element. tion may indicate various opinions as to a path associated The Entry Index and Entry Tag elements are described with a set of time interval thumbnails, the opinions submit above and indicate that a metadata provider may submit data 30 ted by multiple metadata providers. indicating that more than video trailer associated with the The MSI element is described above and identifies the media content, or a composite description may indicate metadata provider. various opinions as to a path associated with a video trailer, The Date Time element is described above and indicates the opinions submitted by multiple metadata providers. the date and time at which the data was received. The MSI element is described above and identifies the 35 XPath Script Or Lyrics metadata provider. The XPath Script Or Lyrics metadata category is used to The Date Time element is described above and indicates store a file path that indicates where a script or lyrics associated with the media content is stored. An example the date and time at which the data was received. entity within the XPath Script Or Lyrics metadata category XPath Chapter Thumbnails 40 has the following elements: The XPath Chapter Thumbnails metadata category is used to store a file path that indicates where a set of chapter thumbnails associated with the media content is stored. Value MSI Thumbnails are small pictures obtained from video frame Entry Index Date Time captures. In this case they are used to provide a pictorial 45 representation as to what the particular chapter of the video Entry Tag is about. An example entity within the XPath Chapter Thumbnails metadata category has the following elements: The Value element stores a file path that identifies the storage location of a file containing a script or lyrics asso 50 ciated with the media content. The Entry Index and Entry Tag elements are described Value MSI Entry Index Date Time above and indicate that a metadata provider may submit Entry Tag paths to more than one file of Script or lyrics associated with the media content, or a composite description may indicate various opinions as to a path associated with a script or The Value element stores a file path that identifies the 55 lyrics, the opinions Submitted by multiple metadata provid storage location of a set of chapter thumbnails associated CS. with the media content. The MSI element is described above and identifies the The Entry Index and Entry Tag elements are described metadata provider. above and indicate that a metadata provider may submit data 60 The Date Time element is described above and indicates indicating that more than one set of chapter thumbnails the date and time at which the data was received. associated with the media content, or a composite descrip XPath Original Storyboard tion may indicate various opinions as to a path associated The XPath Original Storyboard metadata category is used with chapter thumbnails, the opinions submitted by multiple to store a file path that indicates where an original storyboard metadata providers. 65 associated with the media content is stored. An example The MSI element is described above and identifies the entity within the XPath Original Storyboard metadata entity metadata provider. has the following elements: US 7,363,649 B2 70 The Schema Version Used element stores a number that indicates the version of an XML Schema associated with the XML file. Value Entry Tag Date Time MPI Entry Index MSI Each person that may be associated with media content is assigned a unique media person identifier (MPI), Such as The Value element stores a file path that identifies the MPI (1), MPI (2), ..., and MPI (N), that is associated with storage location of a file containing an original storyboard descriptive data related to the person received from one or associated with the media content. more metadata providers 302. The MPI entity stores one The Entry Index and Entry Tag elements are described 10 instance of one element, which is the MPI. above and indicate that a metadata provider may submit Person Category 1, Person Category 2 . . . . Person paths to more than one file containing an original storyboard Category (N) associated with the media content, or a composite descrip Media person data is structured according to categories of tion may indicate various opinions as to a path associated data that may be associated with a person who is associated with an original storyboard, the opinions Submitted by 15 with media content. These categories are represented in FIG. multiple metadata providers. 18 as Person Category 1, Person Category 2, . . . . Person The MSI element is described above and identifies the Category (N). metadata provider. Media content description system 304 may receive media The Date Time element is described above and indicates person data associated with a person from a plurality of the date and time at which the data was received. metadata providers 302. To support multiple data values Exemplary Media Person Data Repository associated with each person category, the media person data Media person data repository 414 stores data that repository 414 Supports multiple entities within each person describes persons who are associated with the media con category. Each entity includes one or more associated ele tent. The data repository may be implemented as a relational mentS. database, an object-oriented database, a set of one or more 25 Exemplary Media Person Data Categories data files, one or more XML files based on an XML schema, FIG. 19 illustrates exemplary media person data catego or any other data structure method. For the purposes of this ries. Media person data may include name, gender, marital discussion, an exemplary media person data repository will status, ethnic origin, religion, height, birth date, birth place, be described as an XML file. alive, death date, death cause, citizenship, residence place, FIG. 18 illustrates the structure of media person data 30 related person, biography one line, biography long, official stored in an exemplary media person data repository 414. home page URL, fan site URL, more information URL, XML Person File Details email address, office phone number, home phone number, The XML Person File Details entity is used to store data fax number, XPath person artwork, and XPath person video. associated with the XML file in which the media person data Name is stored. An example XML Person File Details entity has 35 The Name person category is used to store the names the following elements: associated with a person. An example entity in the Name person category has the following elements:

Person Description File Person Description Creator Organization 40 Version Given Name Nick Name Or Known As Entry Tag Date Time Person Description Language Used For Person Description Family Name Prefix MSI Created First Middle Name PostScript Date Time Person Description Creator Schema Version Used Secon Middle Name Name Type Language Person Third Middle Name Entry Index 45 The Person Description File Version element stores a The Given Name element stores the person’s first name. number that indicates the version of the file. As data is added The Family Name element stores the person’s last name. to a person description file over time, multiple versions of The First Middle Name element stores the person’s first the file may be stored. 50 middle name. The DateTime Person Description Created element stores The Second Middle Name and Third Middle Name ele the date and time that the file was created. ments store additional middle names associated with the The Person Description Creator Person element stores the person. name of the person that created the file. The Nick Name Or Known. As element stores a nickname The Person Description Creator Organization element 55 or alias associated with the person. stores the name of an organization that created the file. The Prefix element stores a prefix to the person's name, The Language Used For Person Description element Such as “Dr. stores a value that indicates the language in which the media The Postscript element stores a postscript to the person’s person description data is provided. As described with name, such as “Jr.” reference to the Language Used For Content Description 60 The Name Type element stores an indicator of the type of element of the XML File Details entity in the content name described. Example name types may include: description metadata repository 412 (see discussion relating Unknown, Current Name, Birth Name, Maiden Name, to FIG. 5), in an exemplary system, the value that is stored Previous Name, Alternate Name, and Other. in the Language Used For Person Description element is a As in entities of the content description metadata reposi combination of a language code and name according to ISO 65 tory categories, the Entry Index and Entry Tag elements are 639. An example list of language field values is given in elements in several person category entities and Support Table 1. multiple entity entries from one metadata provider, or in a

US 7,363,649 B2 73 74 The Value element stores the height of the person. Alive The Entry Index and Entry Tag elements are described The Alive person category is used to store an indicator of above and indicate that a metadata provider may submit whether or not the person is still living. An example entity multiple opinions as to the height of the person, or a within the Alive person category has the following elements: composite description may indicate various opinions as to the height, the opinions Submitted by multiple metadata providers. Value MSI The MSI element is described above and identifies the Entry Index Date Time metadata provider. Entry Tag The Date Time element is described above and indicates 10 the date and time at which the data was received. The Value element stores an indicator of whether the Birth Date person is still living. Example values may include: The Birth Date person category is used to store data Unknown, True, and False. indicating when the person was born. An example entity 15 The Entry Index and Entry Tag elements are described within the Birth Date person category has the following above and indicate that a metadata provider may submit elements: multiple opinions as to whether or not the person is living, or a composite description may indicate various opinions as to whether the person is alive, the opinions submitted by Year Entry Tag multiple metadata providers. Specific Date MSI The MSI element is described above and identifies the Entry Index Date Time metadata provider. The Date Time element is described above and indicates The Year element stores the year in which the person was the date and time at which the data was received. born. 25 Death Date The Specific Date element stores the calendar date on The Death Date person category is used to store data which the person was born. identifying when the person died (if the Alive metadata The Entry Index and Entry Tag elements are described category indicates that the person is not alive). An example above and indicate that a metadata provider may submit entity within the Death Date person category has the fol multiple opinions as to the birth date associated with the 30 lowing elements: person, or a composite description may indicate various opinions as to the birth date, the opinions submitted by multiple metadata providers. Value MSI The MSI element is described above and identifies the Entry Index Date Time metadata provider. 35 Entry Tag The Date Time element is described above and indicates the date and time at which the data was received. The Value element stores the date on which the person Birth Place died. The Birth Place person category is used to store data 40 The Entry Index and Entry Tag elements are described indicating where the person was born. An example entity above and indicate that a metadata provider may submit within the Birth Place person category has the following multiple opinions as to the date of death associated with the elements: person, or a composite description may indicate various opinions as to the date of death, the opinions Submitted by 45 multiple metadata providers. Country Entry Tag The MSI element is described above and identifies the Place Details MSI metadata provider. Entry Index Date Time The Date Time element is described above and indicates the date and time at which the data was received. The Country element stores an indicator that identifies the 50 Death Cause country in which the person was born. Example values are The Death Cause person category is used to store data given in Table 9. identifying what caused the person's death (if the Alive The Place Details element stores additional text describ metadata category indicates that the person is not alive). An ing the location where the person was born. Examples may example entity within the Death Cause person category has include the name of the city or town, or the name of the 55 the following elements: hospital in which the person was born. The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit Value MSI multiple opinions as to the birth place of the person, or a Entry Index Date Time composite description may indicate various opinions as to 60 Entry Tag the birth place, the opinions submitted by multiple metadata providers. The Value element stores text that describes the cause of The MSI element is described above and identifies the the person's death. metadata provider. 65 The Entry Index and Entry Tag elements are described The Date Time element is described above and indicates above and indicate that a metadata provider may submit the date and time at which the data was received. multiple opinions as to the cause of death associated with the US 7,363,649 B2 75 76 person, or a composite description may indicate various the person. An example entity within the Related Person opinions as to the cause of death, the opinions Submitted by person category has the following elements: multiple metadata providers. The MSI element is described above and identifies the metadata provider. Related MPI Entry Tag The Date Time element is described above and indicates Related Name MSI the date and time at which the data was received. Relationship Date Time Citizenship Entry Index The Citizenship person category is used to store data that identifies a country in which the person is a citizen. An 10 The Related MPI element stores the media person iden example entity within the Citizenship person category has tifier (MPI) assigned to another person who is in some way the following elements: related to the person. The Related Name element stores the name of the related person. 15 Value MSI The Relationship element stores an indicator that identi Entry Index Date Time fies how the related person is related to the person. Example Entry Tag values are given in Table 21. The Value element stores in indicator identifying a coun TABLE 21 try in which the person is a citizen. Example values are given Unknown Child Of Biographee in Table 9. Spouse Of Biographee Parent Of Biographee Romantically Linked Same Family AS Biographee The Entry Index and Entry Tag elements are described Previously Romantic With Worked With above and indicate that a metadata provider may submit Ex Spouse Of Biographee Group That Had Biographee multiple countries of citizenship associated with the person, 25 Sibling Other or a composite description may indicate various opinions as to the citizenship, the opinions Submitted by multiple meta The Entry Index and Entry Tag elements are described data providers. above and indicate that a metadata provider may submit The MSI element is described above and identifies the multiple relationships between the person and a related metadata provider. 30 person, or a composite description may indicate various The Date Time element is described above and indicates opinions as to a relationship, the opinions Submitted by the date and time at which the data was received. multiple metadata providers. Residence Place The MSI element is described above and identifies the The Residence Place person category is used to store data metadata provider. identifying where the person lives. An example entity within 35 The Date Time element is described above and indicates the Residence Place person category has the following the date and time at which the data was received. elements: Biography One Line The Biography One Line person category is used to store a one line biographical description of the person. An 40 Country Entry Index example entity within the Biography One Line person State Entry Tag category has the following elements: City MSI Residence Detail Date Time

45 Value MSI The Country element stores in indicator identifying a Entry Index Date Time country in which the person resides. Example values are Entry Tag Language given in Table 9. The State element stores a value that indicates a state in The Value element stores a short biography of the person. which the person resides. 50 The Entry Index and Entry Tag elements are described The City element stores the name of a city in which the above and indicate that a metadata provider may submit person resides. multiple one-line biographies of the person, or a composite The Residence Detail element is used to store additional description may indicate various biographies Submitted by details that describe the person’s residence. multiple metadata providers. The Entry Index and Entry Tag elements are described 55 The MSI element is described above and identifies the above and indicate that a metadata provider may submit metadata provider. multiple places of residence associated with the person, or a The Date Time element is described above and indicates composite description may indicate various opinions as to the date and time at which the data was received. the place of residence, the opinions submitted by multiple The Language element stores an identifier of the language metadata providers. 60 in which the one-line biography is written. A standard set of The MSI element is described above and identifies the language identifiers may be used. An example of Such a metadata provider. standard set of language identifiers is shown in Table 1. The Date Time element is described above and indicates Biography Long the date and time at which the data was received. The Biography Long person category is used to store a Related Person 65 longer biographical description of the person. An example The Related Person person category is used to store data entity within the Biography Long person category has the identifying another individual who is in Some way related to following elements: US 7,363,649 B2 78 More Information URL The More Information URL person category is used to Value MSI store an address of a web site that provides additional Entry Index Date Time information about the person. An example entity within the Entry Tag Language More Information URL person category has the following elements: The Value element stores a long biography of the person. The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit Value MSI multiple biographies of the person, or a composite descrip 10 Entry Index Date Time tion may indicate various biographies Submitted by multiple Entry Tag metadata providers. The MSI element is described above and identifies the The Value element stores a URL that is associated with a metadata provider. 15 web site that provides additional information about the The Date Time element is described above and indicates person. the date and time at which the data was received. The Entry Index and Entry Tag elements are described The Language element stores an identifier of the language above and indicate that a metadata provider may submit in which the biography is written. A standard set of language multiple addresses to informational web sites associated identifiers may be used. An example of Such a standard set with the person, or a composite description may indicate of language identifiers is shown in Table 1. various URLs submitted by multiple metadata providers. Official Home Page URL The MSI element is described above and identifies the The Official Home Page URL person category is used to metadata provider. store an address of a web site that is officially associated The Date Time element is described above and indicates with the person. An example entity within the Official Home 25 the date and time at which the data was received. Page URL person category has the following elements: Email Address The Email Address person category is used to store an email address associated with the person. An example entity Value MSI within the Email Address person category has the following Entry Index Date Time 30 elements: Entry Tag

The Value element stores a URL that is associated with a Value MSI web site that is officially associated with the person. Entry Index Date Time The Entry Index and Entry Tag elements are described 35 Entry Tag above and indicate that a metadata provider may submit multiple URLs of home pages associated with the person, or a composite description may indicate various opinions as to The Value element stores an email address that is asso the official home page, the opinions Submitted by multiple ciated with the person. metadata providers. 40 The Entry Index and Entry Tag elements are described The MSI element is described above and identifies the above and indicate that a metadata provider may submit metadata provider. multiple email addresses associated with the person, or a The Date Time element is described above and indicates composite description may indicate various email addresses the date and time at which the data was received. submitted by multiple metadata providers. Fan Site URL 45 The MSI element is described above and identifies the The Fan Site URL person category is used to store an metadata provider. address of a web site that is associated with fans of the The Date Time element is described above and indicates person. An example entity within the Fan Site URL person the date and time at which the data was received. category has the following elements: Office Phone Number 50 The Office Phone Number person category is used to store a work phone number associated with the person. An example entity within the Office Phone Number person Value MSI Entry Index Date Time category has the following elements: Entry Tag 55

The Value element stores a URL that is associated with a Value MSI web site that is associated with fans of the person. Entry Index Date Time The Entry Index and Entry Tag elements are described Entry Tag above and indicate that a metadata provider may submit 60 multiple fan site web addresses associated with the person, The Value element stores an office phone number that is or a composite description may indicate various URLS associated with the person. submitted by multiple metadata providers. The Entry Index and Entry Tag elements are described The MSI element is described above and identifies the above and indicate that a metadata provider may submit metadata provider. 65 multiple work phone numbers associated with the person, or The Date Time element is described above and indicates a composite description may indicate various phone num the date and time at which the data was received. bers submitted by multiple metadata providers. US 7,363,649 B2 79 80 The MSI element is described above and identifies the resolution. The Size 192x256 element stores a file path that metadata provider. identifies the storage location of artwork associated with the The Date Time element is described above and indicates person that is 192x256 pixels in resolution. the date and time at which the data was received. The Size 720x960 element stores a file path that iden Home Phone Number tifies the storage location of artwork associated with the The Home PhoneNumber person category is used to store person that is 720x960 pixels in resolution. a home phone number associated with the person. An The Entry Index and Entry Tag elements are described example entity within the Home Phone Number person above and indicate that a metadata provider may submit data category has the following elements: indicating a plurality of artwork associated with the person, 10 or a composite description may indicate various paths. Sub mitted by multiple metadata providers. The MSI element is described above and identifies the Value MSI Entry Index Date Time metadata provider. Entry Tag The Date Time element is described above and indicates 15 the date and time at which the data was received. XPath Person Video The Value element stores a home phone number that is The XPath Person Video person category is used to store associated with the person. a file path where a video associated with the person is stored. The Entry Index and Entry Tag elements are described An example entity within the XPath Person Video person above and indicate that a metadata provider may submit category has the following elements: multiple home phone numbers associated with the person, or a composite description may indicate various phone num bers submitted by multiple metadata providers. The MSI element is described above and identifies the File Path Entry Index metadata provider. Resolution Width Entry Tag 25 Resolution Height MSI The Date Time element is described above and indicates Bitrate KBaud Date Time the date and time at which the data was received. Fax Number The Fax Number person category is used to store a fax The File Path element stores a file path that identifies the number associated with the person. An example entity 30 storage location of a video associated with the media con within the Fax Number person category has the following tent. elements: The Resolution Width element stores an indicator of the resolution width of the video stored at the location indicated by the File Path element. The Resolution Height element stores an indicator of the Value MSI 35 Entry Index Date Time resolution height of the video stored at the location indicated Entry Tag by the File Path element. The Bitrate KBaud element stores an indicator of the bitrate of the video stored at the location indicated by the The Value element stores a fax number that is associated File Path element. with the person. 40 The Entry Index and Entry Tag elements are described The Entry Index and Entry Tag elements are described above and indicate that a metadata provider may submit data above and indicate that a metadata provider may submit indicating more than one video associated with the person, multiple fax numbers associated with the person, or a or a composite description may indicate various paths to composite description may indicate various fax numbers videos associated with the person, the paths submitted by submitted by multiple metadata providers. 45 multiple metadata providers. The MSI element is described above and identifies the The MSI element is described above and identifies the metadata provider. metadata provider. The Date Time element is described above and indicates The Date Time element is described above and indicates the date and time at which the data was received. 50 the date and time at which the data was received. XPath Person Artwork Exemplary Metadata Provider Data Repository The XPath Person Artwork person category is used to Metadata provider data repository 416 stores data related store file paths where artwork associated with the person is to content description metadata providers 302. The data stored. An example entity within the XPath Person Artwork repository may be implemented as a relational database, an person category has the following elements: 55 object-oriented database, a set of one or more data files, one or more XML files based on an XML schema, or any other data structure method. For the purposes of this discussion, Size 48x64 Entry Index an exemplary metadata provider data repository will be Size 96x128 Entry Tag described as an XML file. Size 192x256 MSI 60 FIG. 20 illustrates the structure of metadata provider data Size 720x960 Date Time stored in an exemplary metadata provider data repository 416. For each metadata provider 302, metadata provider data The Size 48x64 element stores a file path that identifies repository 416 stores an MSI, identifying data, and one or the storage location of artwork associated with the person more metadata category trust levels. that is 48x64 pixels in resolution. The Size 96x128 element 65 MSI stores a file path that identifies the storage location of Each media description metadata provider 302 is assigned artwork associated with the person that is 96x128 pixels in a unique metadata source identifier (MSI), such as MSI (1), US 7,363,649 B2 81 82 MSI (2), . . . . and MSI (N). Example media description a metadata category. Additionally, alternate implementations metadata providers 302 may include companies that have a are contemplated that relate provider trust levels with dif content identifier Scheme, companies that provide attributes ferent groupings of metadata categories. In yet another and genre categorizations associated with media content, implementation, a provider may be assigned a single trust and companies and individuals who provide critic reviews. level that doesn’t vary across metadata categories. The MPI entity stores one instance of one element, which is Title is the trust level associated with metadata in the title the MPI. metadata category (see FIG. 7) received from the metadata Identifying Data provider. As shown in FIG. 20, metadata provider data repository Episode is the trust level associated with metadata in the 416 stores identifying data that is associated with each 10 episode name metadata category (see FIG. 7) received from metadata provider 302. FIG. 21 illustrates examples of the metadata provider. identifying data that may be stored in association with a Version is the trust level associated with metadata in the metadata provider 302. Identifying data may include com version detail metadata category (see FIG. 7) received from pany name, company URL, Scheme Details URL, Email the metadata provider. Contact 1, Email Contact 2, Email Contact 3, Address Line 15 Album is the trust level associated with metadata in the 1, Address Line 2, Address Town, Address State, and album metadata category (see FIG. 7) received from the Address Zip Code. metadata provider. Company name is the name of the metadata provider 302. Parts is the trust level associated with metadata in the part Company URL is a website address associate with the detail metadata category (see FIG. 7) received from the metadata provider. metadata provider. Scheme details URL is an address associated with a Artists is the trust level associated with metadata in the website that details a content identifier scheme used by the person metadata category (see FIG. 8) received from the metadata provider. metadata provider. The artists trust level applies to those Email contact 1, email contact 2, and email contact 3 are person metadata category entities in which the Person Role email addresses associated with the metadata provider. 25 element is one of General Participant, Actor, Actor Group, Address line 1, address line 2, address town, address state, Music Artist, Music Artist Keyboard, Music Artist and address Zip code are the parts of the metadata provider Drummer, Music Artist Guitarist, Music Artist Lead mailing address. Singer, Music-Artist Backing Singer, Music Band, Host, Metadata Category Trust Level 1-Metadata Category Anchor, News Reporter, Interviewer, Performer, Narrator, Trust Level (N) 30 Dancer, Animator, Graphics Artist, Contestant, or Sub As shown in FIG. 20, metadata provider data repository ject Of Program. 416 stores metadata category trust levels that are associated Director is the trust level associated with metadata in the with each metadata provider 302. The trust level for a person metadata category (see FIG. 8) received from the metadata category that is associated with a provider indi metadata provider. The director trust level applies to those cates the level to which metadata in that category received 35 person metadata category entities in which the Person Role from that provider is to be trusted as accurate. For example, element is one of Director, Assistant Director, Art Direc a metadata provider may be highly trusted to provide tor, or Technical Direction. accurate critic reviews of media content, but have a very low Producer is the trust level associated with metadata in the trust level associated with metadata describing technical person metadata category (see FIG. 8) received from the details associated with media content. In an exemplary 40 metadata. The producer trust level applies to those person embodiment, the trust level is stored as an integer value metadata category entities in which the Person Role element between 0 and 100, with 0 being the lowest trust level and is one of: Producer, Executive Producer, Production Man 100 being the highest trust level. ager, Production Designer, and Production Assistant. In an alternate embodiment, each metadata provider is Editor is the trust level associated with metadata in the assigned Stack level ratings for metadata categories. For 45 each metadata category, the providers are assigned an person metadata category (see FIG. 8) received from the ordered rank. For example, if there are five metadata pro metadata provider. The editor trust level applies to those viders, then for each metadata category, each metadata person metadata category entities in which the Person Role provider is assigned a stack level rank between 1 and 5. element is Editor or Assistant Editor where 1 is the highest rank. 50 Script writer is the trust level associated with metadata in Alternate implementations are contemplated, and may the person metadata category (see FIG. 8) received from the include other methods for ranking or ordering data received metadata provider. The script writer trust level applies to from metadata providers 302. those person metadata category entities in which the Person FIG. 22 illustrates examples of metadata category trust Role element is Script Writer. levels that may be stored in association with a metadata 55 Lyrics is the trust level associated with metadata in the provider 302. Metadata category trust levels may include person metadata category (see FIG. 8) received from the trust levels associated with title, episode, version, parts, metadata provider. The lyrics trust level applies to those artists, director, producer, editor, Script writer, lyrics writer, person metadata category entities in which the Person Role music composer, location date, duration, format, genre element is Lyrics Writer. degrees, genre program type, genre intent, genre target 60 Music composer is the trust level associated with meta audience, genre attributes, review, censor parental ratings, data in the person metadata category (see FIG. 8) received extended censor ratings, origination, features, copyright, from the metadata provider. The music composer trust level textual description, and links. applies to those person metadata category entities in which In this implementation, a metadata category trust level the Person Role element is Music Composer. may apply to a plurality of metadata categories. In an 65 In alternate embodiments, trust levels may also be asso alternate implementation, a metadata provider 302 may be ciated with metadata that describes other persons associated assigned trust levels such that each trust level corresponds to with the media content. US 7,363,649 B2 83 84 Location is the ranking associated with metadata in the text Page Number metadata categories (see FIG. 15) capture location metadata category (see FIG. 14) received received from the metadata provider. from the metadata provider. Made For is the trust level associated with metadata in the Date is the trust level associated with metadata in the Made For metadata category (see FIG. 14) received from the availability date metadata category (see FIG. 10) received metadata provider. from the metadata provider. Budget is the trust level associated with metadata in the Duration is the trust level associated with metadata in the Budget metadata category (see FIG. 14) received from the content duration metadata category (see FIG. 10) received metadata provider. from the metadata provider. Box Office is the trust level associated with metadata in Format is the trust level associated with metadata in the 10 the Box Office metadata category (see FIG. 14) received format metadata category (see FIG. 9) received from the from the metadata provider. metadata provider. Production Company is the trust level associated with Genre degrees is the trust level associated with metadata metadata in the Production Company, Principal Country Of in the genre degrees metadata category (see FIG.9) received Main Production Company, and URL Production from the metadata provider. 15 Genre program type is the trust level associated with Company metadata categories (see FIG. 14) received metadata in the genre program type metadata category (see from the metadata provider. FIG. 9) received from the metadata provider. Distribution Company, is the trust level associated with Genre intent is the trust level associated with metadata in metadata in the Distribution Company metadata category the genre intent metadata category (see FIG. 9) received (see FIG. 14) received from the metadata provider. from the metadata provider. Language is the trust level associated with metadata in the Genre target audience is the trust level associated with Language Data metadata categories 620, listed in FIG. 16. metadata in the target audience metadata category (see FIG. received from the metadata provider. 9) received from the metadata provider. Paths is the trust level associated with metadata in the Genre attributes is the trust level associated with metadata 25 Schema Implementation Details metadata categories 622, in the year set, era set, location portrayed, from the book, listed in FIG. 17, received from the metadata provider. degree of true story, degree of animated 2D, degree of Methods for Storing Media Content Description Metadata animated 3D. degree of puppet characters, degree of inter FIG. 23 illustrates a process for storing media content national event, degree of Sophistication, genre music, and description metadata. The process is illustrated as a set of genre photo or home movie metadata categories (see FIG.9) 30 operations shown as discrete blocks. The process may be received from the metadata provider. implemented in any Suitable hardware, Software, firmware, Review is the trust level associated with metadata in the or combination thereof. The order in which the operations critic reviews metadata category (see FIG. 11) received from are described is not to be construed as a limitation. For the metadata provider. discussion purposes and where appropriate, the process will Popularity is the trust level associated with metadata in 35 be described with reference to the architecture 300 of FIG. the popularity metadata category (see FIG. 11) received 3 and the system 304 of FIG. 4. from the metadata provider. At block 2302, a media content description system 304 Broadcast Ubiquity is the trust level associated with receives metadata from a metadata provider 302. For pur metadata in the broadcast ubiquity metadata category (see poses of this discussion, it is assumed that the metadata FIG. 11) received from the metadata provider. 40 received pertains to a particular media content, such as a Censor parental ratings is the trust level associated with movie. If the media content description system receives metadata in the censor parental rating metadata category metadata describing a plurality of media content, then the (see FIG. 12) received from the metadata provider. process is repeated for the metadata associated with each Extended censor ratings is the trust level associated with media content described. metadata in the extended censor rating adult content, 45 extended censor rating bad language, extended censor rating At block 2304, the media content description system 304 violence, extended censor rating nudity, and extended censor determines whether the received metadata includes metadata rating drug abuse metadata categories (see FIG. 12) received that describes one or more persons associated with the media from the metadata provider. content. Given a movie as the media content, persons that Origination is the trust level associated with metadata in 50 may be described who are associated with the media content the Technical Details metadata categories 614, listed in FIG. may include actors, directors, producers, and screenwriters. 13, received from the metadata provider. If the received metadata does not include metadata describ Features is the trust level associated with metadata in the ing one or more persons associated with the media content ATVEF Data Enhancements, Educational Commands, Edu (e.g., the no branch from block 2304), the process continues at block 2312. cational Informational Enhancements, Multiple Camera 55 Angles, Multiple Story Lines, Phone In, Associated Phone If the received metadata includes metadata describing one Number, and VCR Programming Code metadata categories or more persons associated with the media content (e.g., the (see FIG. 15) received from the metadata provider. yes branch from block 2304), then, at block 2306, the media Copyright is the trust level associated with metadata in the content description system 304 identifies data that was copyright metadata category (see FIG. 14) received from the 60 received that is descriptive of a person associated with the metadata provider. media content. Textual description is the trust level associated with At block 2308, the media content description system 304 metadata in the plot one line abstract, plot long abstract, stores the data describing the person in a media person data keywords, and content note metadata categories (see FIG. repository 414. The process executed by the media content 10) received from the metadata provider. 65 description system to store the data in the media person data Links is the trust level associated with metadata in the repository is described below in more detail with reference URL Enhancements, URL More Info, and Associated Tele to FIG. 24. US 7,363,649 B2 85 86 At block 2310, the media content description system 304 egory was already stored in the media person data repository determines whether additional persons associated with the 414 and associated with the person described by the received media content are described by the received metadata. metadata. If additional persons are described by the received meta If there was already an entity in the identified person category stored in association with the described person data (e.g., the “yes” branch from block 2310), then the (e.g., the “yes” branch from block 2408), then the media process is repeated beginning at block 2306. content description system 304 generates a composite entity After all of the received data describing persons associ based on all of the entities stored in the identified person ated with the media content is stored in the media person category associated with the person (block 2410). The data repository (e.g., the “no branch from block 2310), or 10 process for generating a composite entity is described below if none of the received metadata describes persons associ in more detail with reference to FIG. 26. ated with the media content (e.g., the “no branch from At block 2412, the media content description system 304 block 2304), the media content description system 304 determines whether the received metadata that describes the populates a content description metadata repository 412 with person includes data that is associated with another person the received media content description metadata (block 15 category. 2312). The process executed by the media content descrip If there is data associated with additional person catego tion system to store the data in the content description ries, the process repeats, beginning at block 2404 (e.g., the metadata repository is described below in more detail with “yes” branch from block 2412). reference to FIG. 25. Populating Content Description Metadata Repository Populating Media Person Data Repository FIG. 25 illustrates a process for storing metadata that FIG. 24 illustrates a process for storing metadata that describes the media content. The process is illustrated as a describes a person associated with media content. The set of operations shown as discrete blocks. The process may process is illustrated as a set of operations shown as discrete be implemented in any suitable hardware, software, firm blocks. The process may be implemented in any Suitable ware, or combination thereof. The order in which the opera hardware, software, firmware, or combination thereof. The 25 tions are described is not to be construed as a limitation. For order in which the operations are described is not to be discussion purposes and where appropriate, the process will construed as a limitation. For discussion purposes and where be described with reference to the architecture 300 of FIG. appropriate, the process will be described with reference to 3 and the system 304 of FIG. 4. the architecture 300 of FIG.3 and the system 304 of FIG. 4. At block 2502, a media content description system 304 At block 2402, a media content description system 304 30 determines a unique identifier associated with the media determines a unique identifier associated with the person content described by the received metadata. For example, described by the received metadata. For example, if the media content description system 304 searches content received metadata describes the director of a movie, media description metadata repository 412 for data describing the content description system 304 searches media person data same media content described by the received metadata repository 414 for data describing the same person described 35 (e.g., the movie described by the received metadata may by the received metadata. The media person data repository already be described by data stored in the content descrip 414 may already contain a description of the director of the tion metadata repository 412). If the media content is already movie. For example, a description of the movie may have described by data in the content description metadata reposi already been received from another content description tory 412, then media content description system 304 deter metadata provider 302 that included a description of the 40 mines the media content identifier (MCID) associated with director. Alternatively, the director of the movie may have the already stored data as the MCID associated with the directed another movie that is described in the media content media content described by the received metadata. If the description system 304, or the director of the movie may media content is not already identified in the content also be an actor in another movie or television show description metadata repository 412, then media content described by data stored in the media content description 45 description system 304 generates a new MCID to associate system 304. If the person is already described by data in the with the media content described by the received metadata. media person data repository, then the media content At block 2504, the media content description system 304 description system determines the media person identifier identifies a metadata category associated with the received (MPI) that is associated with the already stored data as the metadata (e.g., a metadata category as described with ref MPI associated with the person described by the received 50 erence to FIGS. 5-17). Metadata categories may include metadata. If the person is not already identified in the media categories that describe genre, descriptions, popularity, cen person data repository, then media content description sys Sor ratings, technical details, or any of the other metadata tem generates a new MPI to identify the person described by categories listed in FIGS. 6-17. the received metadata. At block 2506, the media content description system 304 55 stores the received metadata that applies to the identified At block 2404, the media content description system 304 data category as a metadata category entity in content identifies a category associated with the received metadata description metadata repository 412. that describes a person (e.g., a person category as described At block 2508, the media content description system 304 with reference to FIGS. 18 and 19). Person categories may determines whether an entity in the identified metadata include a name, gender, marital status, ethnic origin, or any 60 category was already stored in the content description meta of the other person categories listed in FIG. 19. data repository and associated with the media content At block 2406, the media content description system 304 described by the received metadata. stores the received metadata that applies to the identified If there was already an entity in the identified metadata person category as a person category entity in media person category stored in association with the described media data repository 414. 65 content (e.g., the “yes” branch from block 2508), then the At block 2408, the media content description system 304 media content description system 304 generates a composite determines whether an entity in the identified person cat entity based on all of the entities stored in the identified US 7,363,649 B2 87 88 metadata category associated with the media content (block represent a plurality of metadata providers by a provider 2510). The process for generating a composite entity is class in a schema independent of a particular broadcast described below in more detail with reference to FIG. 26. instance; At block 2512, the media content description system 304 represent a plurality of media content descriptions in a determines whether the received metadata that describes the media content class of the schema; and media content includes data that is associated with another create an association to associate media content descrip metadata category. tions in the media content class of the schema with If there is data associated with additional metadata cat metadata providers in the provider class of the schema. egories, the process repeats, beginning at block 2504 (e.g., 2. The one or more computer-readable media as recited in the “yes” branch from block 2512). 10 claim 1, wherein the provider class represents at least one of Generating a Composite Entity a provider identifier, a company name, a company URL, a FIG. 26 illustrates a process for generating a composite scheme details URL, an email contact, an address, or a metadata entity. This process applies to generating compos provider trust level. ite entities in the content description metadata repository 3. The one or more computer-readable media as recited in 412 or the media person data repository 414. The process is 15 claim 2, wherein the provider trust level indicates at least illustrated as a set of operations shown as discrete blocks. one of a ranking of the metadata providers or a degree to The process may be implemented in any Suitable hardware, which metadata from a metadata provider is trusted. software, firmware, or combination thereof. The order in 4. The one or more computer-readable media as recited in which the operations are described is not to be construed as claim 2, wherein the provider trust level is associated with a limitation. For discussion purposes and where appropriate, the at least a portion of the media content class. the process will be described with reference to the architec 5. The one or more computer-readable media as recited in ture 300 of FIG. 3 and the system 304 of FIG. 4. claim 2, wherein the provider trust level is associated with At block 2602, a media content description system 304 at least one of a title, an episode, a version, a part, an artist, determines an element of an entity within the metadata a director, a producer, an editor, a script writer, a lyrics category for which a composite entity is being generated. 25 writer, a music composer, a location, a date, a duration, a At block 2604, the media content description system 304 format, a genre degree, a genre program type, a genre intent, determines the distinct values of the element that are stored a target audience description, a genre attribute, a review, a within the existing category entities. censor rating, an extended censor rating, an origination, a At block 2606, the media content description system 304 feature, a copyright, a textual description, and a link. determines whether there is more than one distinct value 30 6. The one or more computer-readable media as recited in stored for the element. claim 1, wherein the a media content description comprises If there is only one distinct value stored for the element at least one of a media content identifier, content identifying (e.g., the “no branch from block 2606), the media content data, an associated person, genre data, content description description system 304 sets the value for the element within data, a critic review, a popularity, a broadcast ubiquity, the composite entity to the one distinct value already stored 35 censor rating data, technical details data, production details for the element. The process then continues at block 2614. data, enhancement data, language data, or associated media. However, if there is more than one distinct value stored 7. The one or more computer-readable media as recited in for the element (e.g., the “yes” branch from block 2606), claim 6, wherein the media content identifier comprises: then for each distinct value, the media content description a representation of a title; system 304 determines the sum of the trust levels associated 40 a representation of an episode: with the providers of that value (block 2610). a representation of a version; and At block 2612, the media content description system 304 a representation of a part. sets the value for the element within the composite entity to the distinct value that has the highest associated trust level 8. The one or more computer-readable media as recited in summation, as determined at block 2610. 45 claim 7, wherein the representations of the title, episode, At block 2614, the media content description system 304 version, and part each have an associated significance. determines whether there are additional elements associated 9. The one or more computer-readable media as recited in with an entity of the metadata category. claim 7, wherein the representations of the title, episode If there are additional elements associated with an entity name, version, and part each have an associated significance, 50 and wherein the significance associated with the represen of the metadata category (e.g., the “yes” branch from block tation of the title is greater than the significances associated 2614), then the media content description system 304 con with the representations of the episode, version, and part. tinues the process at block 2602. 10. The one or more computer-readable media as recited CONCLUSION in claim 6, wherein the content identifying data comprises at 55 least one of an alternate content identifier, a title, an episode Although the systems and methods have been described in name, an album name, a version reason, a version descrip language specific to structural features and/or methodologi tion, a part number, or a total number of parts. cal steps, it is to be understood that the invention defined in 11. The one or more computer-readable media as recited the appended claims is not necessarily limited to the specific in claim 6, wherein the associated person comprises at least features or steps described. Rather, the specific features and 60 one of an identifier associated with a person, a name, a steps are disclosed as preferred forms of implementing the gender, a marital status, an ethnic origin, a religion, a height, claimed invention. a birth date, a birth place, an alive indicator, a death date, a death cause, a citizenship, a residence place, a related The invention claimed is: person, a biography, an official home page URL, a fan site 1. One or more computer-readable media comprising 65 URL, a more information URL, an email address, an office computer-executable instructions, that when, executed cause phone number, a home phone number, a fax number, an a computer to: associated artwork, or an associated video. US 7,363,649 B2 89 90 12. The one or more computer-readable media as recited least one of a capture mechanism, an image capture reso in claim 6, wherein the genre data comprises at least one of lution, a video capture temporal rate frequency, a video a genre program type, a genre degree, a genre intent, a target captured using interlace indicator, Sound capture sampling audience description, a year set, an era set, a location data, Sound capture compression data, a camera used, image portrayed, an associated book, a degree of truth, a degree of 5 capture compression data, a recorded live indicator, a black two-dimensional animation, a degree of three-dimensional and white indicator, a silent indicator, post production pro animation, a degree of puppet characters, a degree of inter cessing data, special electronic processing data, an aspect national event, a degree of Sophistication, a music genre, a ratio, an aspect ratio implementation, pan and scan vectors, photo genre, a home movie genre, or a format. an origination note, a stereo sound indicator, or Surround 13. The one or more computer-readable media as recited 10 Sound data. in claim 12, wherein the genre degree comprises at least one 17. The one or more computer-readable media as recited indicator of how well the media content fits into a genre, the in claim 6, wherein the production details data comprises at genre comprising at least one of action, adventure, horror, least one of a made for indicator, a budget, a box office comedy, death, mystery, police involvement, thriller, politi Success indicator, a production company, a distribution cal intrigue, romance, erotica, Science fiction, period setting, 15 company, a principal country associated with a main pro lives drama, sports interest, animal interest, medical interest, duction company, a capture location, copyright data, or a legal interest, religious interest, historical interest, war inter URL associated with a production company. est, epic production, fantasy folklore, musical, western, 18. The one or more computer-readable media as recited monsters, teenage college, ethnic interest, or soap. in claim 6, wherein the enhancement data comprises at least 14. The one or more computer-readable media as recited one of an ATVEF data enhancement indicator, an educa in claim 12, wherein the genre intent comprises at least one tional commands indicator, an educational informational indicator of a degree to which the media content is associ enhancements indicator, a multiple camera angles indicator, ated with an intent, the intent comprising at least one of a multiple story lines indicator, a phone in indicator, a URL education, entertainment, news, information, enrich, enhancements indicator, a more information URL indicator, involve, or sell. 25 an associated phone number, an associated teletext page 15. The one or more computer-readable media as recited number, or a VCR programming code. in claim 12, wherein the format comprises at least one of 19. The one or more computer-readable media as recited video movie cinema released, video drama mini series or TV in claim 6, wherein the language data comprises at least one movie, video sitcom, video soap, video sports, video games of a primary original language, an original segment lan or reality, Video talk show or non sitcom comedy, video 30 guage, a dubbed language, an audio track language, a burnt news or current affairs, video documentary, video kids or in text language, a text track language, a description track other cartoon, video how to or educational, video religious language, or data associated with a sign language track. local access or paid, video specials, video amateur, audio 20. The one or more computer-readable media as recited Song track, audio talking book, audio amateur recording, in claim 6, wherein the associated media comprises at least audio other, photo amateur, photo professional, photo other, 35 one of content artwork, a video trailer, a chapter thumbnail, advert, collection video, collection audio, collection photo, a time interval thumbnail, a script, lyrics, or an original or collection other. storyboard. 16. The one or more computer-readable media as recited in claim 6, wherein the technical details data comprises at