USOO7363649B2 (12) United States Patent (10) Patent No.: US 7,363,649 B2 Marsh (45) Date of Patent: Apr. 22, 2008 (54) MEDIA CONTENT DESCRIPTIONS 6,614,987 B1 9/2003 Ismail et al. 6,651,253 B2 * 1 1/2003 Dudkiewicz et al. ....... T25,114 (75) Inventor: David J. Marsh, Sammamish, WA 6,704,491 B1 3/2004 Revis (US) 6,842,761 B2 1/2005 Diamond et al. 6,973,665 B2 12/2005 Dudkiewicz et al. 2001/004.1980 A1 11/2001 Howard et al. (73) Assignee: Microsoft Corporation, Redmond, WA 2002/0042923 A1 4/2002 Asmussen et al. (US) 2002/0104087 A1 8, 2002 Schaffer et al. 2002fO140719 A1 * 10, 2002 Amir et al. ................. 345,716 (*) Notice: Subject to any disclaimer, the term of this 2002/0147984 A1 10, 2002 Tomsen et al. patent is extended or adjusted under 35 2002/0152463 A1 10, 2002 Dudkiewicz U.S.C. 154(b) by 0 days. (Continued) (21) Appl. No.: 11/119,170 FOREIGN PATENT DOCUMENTS (22) Filed: Apr. 29, 2005 JP 2000 067060 3, 2000 (65) Prior Publication Data (Continued) US 2005/O192987 A1 Sep. 1, 2005 OTHER PUBLICATIONS O O Jasinschi, R.S. et al., “Automatic TV Program Genre Classification Related U.S. Application Data Based on Audio Patterns’, IEEE 2001, 6 pages, Philips Research, (62) Division of application No. 10/125,260, filed on Apr. Scarborough Road, Briarcliff Manor, NY. 10510, USA. 16, 2002, now Pat. No. 7,073,193. (Continued) (51) Int. Clwe PrimaryFIFOF ExaminerEXOFilief CharlesaS RExOS G06F 7/30 (2006.01) Assistant Examiner Belix M Ortiz (52) U.S. Cl. ................... 725/114; 725/109; 707/104.1; (74) Attorney, Agent, or Firm—Lee & Hayes, PLLC 707/100 (58) Field of Classification Search ................ 725/114 (57) ABSTRACT 725/138, 144; 345/716 See application file for complete search history. A media content description system receives media content descriptions from one or more metadata providers. The (56) References Cited media content description system associates each media content description with the metadata provider that provided U.S. PATENT DOCUMENTS the description. The media content description system may 5,969,748 A 10, 1999 Casement et al. generate composite descriptions based on the received 5,973,683 A 10/1999 Cragun et al. media content descriptions. The media content description 5,995,133 A 11, 1999 Kim 6,115,057 A 9, 2000 Kwoh et al. system provides data associated with the media content 6,157,411 A 12/2000 Williams et al. descriptions to one or more program data providers. 6,317,885 B1 11/2001 Fries 6,457,010 B1 9/2002 Eldering et al. 20 Claims, 17 Drawing Sheets Media Cortent 304 Content description description System Metadata Provider(s) Sci. 302 300 y o iders 306 EPG. Database EPG Sewer gontent 308 3.18 Providers Stored Content 322 Content Distribution System 310 Content Data Broadcast Content Server Processor(s) Transmitter 320 328 324 328 Broadcast Network C 312(1) 312(2) 312(N) se o 332 330(1) 33O2) as s es s 334 ess US 7,363,649 B2 Page 2 U.S. PATENT DOCUMENTS Liu et al., “Classifying Video Documents. By Hierarchical Structure Of Video Contents'. The Computer Journal, vol.43, No. 5, 2000, 15 2003/OO93694 A1* 5/2003 Medvinsky et al. ........ T13 201 pageS. 2003/O135857 A1 7/2003 Pendahur et al. Truong et al., “Automatic Genre Identification for Content-Based 2005, 0028194 A1 2/2005 Elenbaas et all Video Categorization', IEEE, 2000, 4 pages. 2005/O192987 A1 9, 2005 Marsh De Jong et al., “Die technische Fachzeitschrift fur Fernsehen, Film 2006, O190966 A1 8, 2006 McKissicket al. und elektronische Medien'. Fernseh-Und Kino-Technik-55, Jahrgang, Nr. Aug. 9, 2001, 10 pages. FOREIGN PATENT DOCUMENTS Mulder, P. “The Integration Of Metadata from Production To JP 2001 134582 5, 2001 Consumer', EBU Technical Review, Sep. 2000, 5 pages. WO WOOOf 40026 T 2000 Evain, J., “TV-Anytime Metadata, A Preliminary Specification On WO WO O1/47257 6, 2001 Schedule”, EBU Technical Review, Sep. 2000, 14 pages, EBU WO WO O2,11445 2, 2002 Technical Department. Herla, Von Siegbert, "Online-Archive-MIT Metadaten Zum Erfolg', OTHER PUBLICATIONS Dec. 15, 2000, 8 pages. Joung, Ye Sun, “A Metadata Repository System For An Efficient Wei et al., “TV Program Classification Based on Face and Text Description Of Visual Multimedia Documents”. Concurrent Engi Processing, 2000, 4 pages; Computer Science Department, Wayne neering, Research and Applications, vol. 9, No. 2, Jun. 2001, 12 University, Detroit MI, Philips Research, Briarcliff Manor, NY. pageS. Taskiran et al., “Discovering Video Structure Using The Psuedo Adami et al., “The ToCAI Description Scheme For Indexing And Semantic Trace'. Proceedings of SPIE, vol. 4315, 2001, 8 pages. Retrieval Of Multimedia Documents', Multimedia Tools And McDonald et al., “Online Television Library: Organisation and Applications, 14, 153-173, 2001. Content Browsing for General Users'. Proceedings of SPIE, vol. 4315, 2001, 9 pages. * cited by examiner U.S. Patent Apr. 22, 2008 Sheet 1 of 17 US 7,363,649 B2 PROGRAM DATA CONTENT DESCRIPTION METADATA 1 STORY LINE DIRECTOR CRITIC OPINIONS REVIEWS RECOMMENDATION S INSTANCE DESCRIPTION METADATA 104 TIME CHANNEL U.S. Patent Apr. 22, 2008 Sheet 2 of 17 US 7,363,649 B2 REVIEWS 204 CRITIC OPINIONS RECOMMENDATIONS 204 STORY LINE RATINGS DIRECTOR ACTORS U.S. Patent Apr. 22, 2008 Sheet 3 of 17 US 7,363,649 B2 Media Content 304 Content Description Description System Metadata Provider(s) 302 Program Data Provider(s) 306 EPG Database EPG Server Content 308 316 318 Provider(s) Content Distribution System 310 Broadcast Content Server Processor(s) Transmitter 320 326 324 | 314 328 Broadcast NetWork 312(1) 312(2) 312(N) U.S. Patent Apr. 22, 2008 Sheet 4 of 17 US 7,363,649 B2 Media Content Description System Metadata Provider Program Data Interface(s) Provider interface(s) Processor(s) O2 404 Operating Media Person Data Repository 10 414 Content Description Metadata Provider Metadata Repository Data Repository 412 416 U.S. Patent Apr. 22, 2008 Sheet 5 of 17 US 7,363,649 B2 Content Description Metadata Repository XML File Details MCID(1) Metadata Category 1 Category 1 Entity (Element 1, Element 2, ...) Category 1 Entity (Element 1, Element 2, ...) Metadata Category 2 Category 2 Entity (Element 1, Element 2, ...) Category 2 Entity (Element 1, Element 2, ...) Metadata Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MCID(2) Metadata Category 1 Category 1 Entity (Element 1, Element 2, ...) Category 1 Entity (Element 1, Element 2, ...) Metadata Category 2 Category 2 Entity (Element 1, Element 2, ...) Category 2 Entity (Element 1, Element 2, ...) Metadata Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MCID(N) U.S. Patent Apr. 22, 2008 Sheet 6 of 17 US 7,363,649 B2 Associated People Genre Data Content Descriptions 604 606 608 102 \, Popularity 1O Censor Ratings 12 ldentifying Data 602 Technical Details 614 Production Details 616 Schema Implementation Language Data Enhancements Details 618 U.S. Patent Apr. 22, 2008 Sheet 7 Of 17 US 7,363,649 B2 identifvind Data Genre Data 602 60 Alternate Content dentifiers Genre Program Type Title Genre Degrees Episode Name Genre intent Album Target Audience Version Detail Year Set Part Detail Era Set Location Portrayed From The Book Degree of True Story Degree of Animated 2D Degree of Animated 3D Degree of Puppet Characters Degree of International Event Associated People Degree of Sophistication Genre Music Genre Photo or Home Movie Content Descriptions 608 Popularity Plot One Line Abstract 1 O Plot Long Abstract Keywords Critic Reviews Content Note Popularity Availability Date Broadcast Ubiquity Content Duration U.S. Patent Apr. 22, 2008 Sheet 8 of 17 US 7,363,649 B2 Censor Ratings 12 Censor Parental Rating Extended Censor Rating Adult Content Extended Censor Rating Bad Language Extended Censor Rating Violence Extended Censor Rating Nudity Extended Censor Rating Drug Abuse Technical Details 14 Capture Mechanism Image Capture Resolution Video Capture Temporal Rate Hz Video Captured Using Interlace Sound Capture Sampling Sound Capture Compression Camera Used Image Capture Compression Recorded Live Black and White Silent Post Production Processing Special Electronic Processing Aspect Ratio Aspect Ratio Implementation Pan and Scan Vectors Origination Note Stereo Sound Surround Sound U.S. Patent Apr. 22, 2008 Sheet 9 Of 17 US 7,363,649 B2 Production Details Language Data Made For Language Primary Original Budget Language Segment Original Box Office Language Dub Production Company Language Audio Track Distribution Company Language Text Burnt in Principal Country of Main Production Company language Text Track Capture Location Language Description Track Copyright Sign Language Track URL Production Company Enhancements 18 ATVEF Data Enhancement Educational Commands Educational Informational Enhancements Schema Implementation Multiple Camera Angles Details 622 Multiple Story Lines Phone in XPath Content ArtWork URL Enhancements XPath Video Trailer URL More info XPath Chapter Thumbnails ASSOciated Phone Number XPath Time interval Thumbnails Associated Teletext Page Number XPath Script Or Lyrics VCR Programming Code XPath Original Storyboard U.S. Patent Apr. 22, 2008 Sheet 10 of 17 US 7,363,649 B2 Media Person Data Repository XML. Person File Details MPI (1) Person Category 1 Category 1 Entity (Element 1, Element 2, ... Category 1 Entity (Element 1, Element 2, ... Person Category 2 Category 2 Entity (Element 1, Element 2, ... Category 2 Entity (Element 1, Element 2, ... Person Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MPI (2) Person Category 1 Category 1 Entity (Element 1, Element 2, ...) Category 1 Entity (Element 1, Element 2, ...) Person Category 2 Category 2 Entity (Element 1, Element 2, ...) Category 2 Entity (Element 1, Element 2, ...) Person Category (N) Category (N) Entity (Element 1, Element 2, ...) Category (N) Entity (Element 1, Element 2, ...) MPI (N) U.S.