USOO7664882B2

(12) United States Patent (10) Patent No.: US 7.664,882 B2 Mohammed et al. (45) Date of Patent: Feb. 16, 2010

(54) SYSTEMAND METHOD FOR ACCESSING 5,420,801 A 5/1995 Dockter et al. MULTIMEDIA CONTENT 5,528,281 A 6/1996 Grady et al. 5,539,886 A 7/1996 Aldred et al. (75) Inventors: Sohail Baig Mohammed, Redmond, 5,546,584 A 8, 1996 Lundin et al. WA (US); Alexandre V. Grigorovitch, 5,574,934 A 11/1996 Mirashrafi et al. Redmond, WA (US); Kirt A. Debique, ... 1 S...

SMSYenlo Park, US, the; Sume Sis. 5,625.4045,675,752 A 10,4/1997 1997 ScottGrady et et al. al. Barde, Redmond, WA (US); Miguel M. 5,712,906 A 1/1998 Grady et al. Valdez, Redmond, WA (US) 5,764,965 A 6/1998 Poimboeufetal. 5,765,011 A 6/1998 Wilkinson et al. (73) Assignee: Microsoft Corporation, Redmond, WA 5,786,814 A 7/1998 Moran et al. (US) 5,802.283 A 9/1998 Grady et al. 5,815,689 A 9, 1998 Shaw et al. (*) Notice: Subject to any disclaimer, the term of this 3. A R Rd et al. sy sys- ungle past listsoyed under 35 5,887,139 A 3/1999 Madison, Jr. et al. Continued (21) Appl. No.: 10/829,503 ( ) FOREIGN PATENT DOCUMENTS (22) Filed: Apr. 22, 2004 EP 0784271 (A2) 7/1997 (65) Prior Publication Data (Continued) US 2005/O188413 A1 Aug. 25, 2005 OTHER PUBLICATIONS Related U.S. Application Data Chatterjee, et al., “Microsoft DirectShow: A New Media Architec ture”, SMPTE J 1, Dec. 1997, pp. 865-871. (60) Provisional application No. 60/546,460, filed on Feb. e Ola CC pp 21, 2004. (Continued) (51) Int. Cl Primary Examiner Vivek Srivastava Goof i 5/16 (2006.01) Assistant Examiner—Ryan J Jakovac (52) U.S. Cl 709/250; 709/231 (74) Attorney, Agent, or Firm Lee & Hayes, PLLC (58) Field of Classification Search ...... 705. li (57) ABSTRACT See application file for complete search history. Systems and methods described herein provide access to (56) References Cited multimedia content of varying types from a variety of sources using a uniform resource locator (URL). U.S. PATENT DOCUMENTS 5,140,437 A 8, 1992 Yonemitsu et al. 48 Claims, 4 Drawing Sheets

20 y

C Media Application 24 Byte 282 3. Stream handlers Media processing Module 22 280 284 objectSelection Module 214 286 Byte Stream Shee Harder Hadler

lists).21 list(s).211.

Transform(s) 22

edia Content Destination 208

Media Content Store 28

Media Content.212 US 7,664,882 B2 Page 2

U.S. PATENT DOCUMENTS 2002/O123997 A1 9/2002 Loy et al. 2002fO158897 A1 10, 2002 Besaw et al. 5,892,767 A 4/1999 Bell et al. 2002/0174425 A1 11/2002 Markelet al. 5,936,643 A 8, 1999 Tindell et al. 2002/0199031 A1* 12/2002 Rust et al...... 709,315 5,987,628 A 11/1999 Von Bokernet al. 2003/0028643 A1 2, 2003 Jabri 5,995,512 A 1 1/1999 Pogue, Jr. 2003/0033424 A1 2/2003 Gould ...... TO9,233 5.996,015 A 1 1/1999 Day et al. 2003/0056029 A1 3/2003 Huang et al. 6,014,706 A 1/2000 Cannon et al. 2003/0093568 A1 5/2003 Deshpande 6,038,625 A 3/2000 Ogino et al. 2003/0095504 A1 5/2003 Ogier 6,044,408 A 3/2000 Engstrometal. 2003/0101.253 A1 5.2003 Saito et al. 6,178,172 B1 1/2001 Rochberger 2003/O123659 A1 7/2003 Forstrom et al. 6,185,612 B 1 2/2001 Jensen et al. 2003/0146915 A1 8/2003 Brook et al. 6, 192,354 B1 2/2001 Bigus et al. 2003. O149772 A1 8, 2003 HSu et al. 6,209,041 B1 3/2001 Shaw et al. 2003. O158957 A1 8, 2003. Abdolsaleh 6,243,753 B1 6/2001 Machin et al. 2003/0177292 A1* 9, 2003 Smirnov et al...... T10.30 6,262,776 B1 7/2001 Griffits 2003/0215214 A1 11/2003 Ma 6,263.486 B1 7/2001 Boezeman et al. 2003/0231867 A1 12/2003 Gates et al. 6,266,053 B1 7/2001 French et al. 2003/02368.92 A1 12/2003 Coulombe 6.279,029 B1 8/2001 Sampat et al. 2003/0236906 A1* 12/2003 KlemetSet al...... TO9,231 6,308,216 B1 10/2001 Goldszmidt et al. 2004/0001106 A1 1/2004 Deutscher et al. 6,317,131 B2 11/2001 Basso et al. 2004/0004631 A1 1/2004 Debique et al. 6,321,252 B1 1 1/2001 Bhola et al. 2004/0031058 A1 2/2004 Reisman 6,343,313 B1 1/2002 Salesky et al. 2004/0042413 A1 3/2004 Kawamura et al. 6,347,079 B1 2/2002 Stephens et al. 2004/0073596 A1 4/2004 Kloninger et al. 6,369,835 B1 4/2002 Lin 2004/OO739 12 A1 4, 2004 Meza 6,385,201 B1 5/2002 Iwata 2004/0080504 A1 4/2004 Salesky et al. 6,389,467 B1* 5/2002 Eyal ...... 709,223 2004/O139157 A1 7/2004 Neely, III et al. 6,430,526 B1 8/2002 Toll 2004/0177162 A1 9, 2004 Wetzel et al. 6,466,971 B1 10/2002 Humpleman et al. 2004/02O7723 A1 10, 2004 Davis et al. 6,536,043 B1 3/2003 Guedalia 2004/0208132 A1 10, 2004 Neulist et al. 6,539,163 B1 3/2003 Sheasby et al. 2004/0220926 A1 11/2004 Lamkin et al. 6,546,426 B1 4/2003 Post 2004/0230659 A1 11, 2004 Chase 6,549,932 B1 4/2003 McNally et al. 2004/0236945 A1 11/2004 Risan et al. 6,581,102 B1 6.2003 Amini et al. 2004/0267778 Al 12/2004 Rudolph et al. 6,594,699 B1 7/2003 Sahai et al. 2004/0267899 A1 12/2004 Rahman et al. 6,594,773 B1 7/2003 Lisitsa et al. 2004/0267953 A1 12/2004 Dunbar et al. 6,618,752 B1 9/2003 Moore et al. 2004/0268224 A1 12/2004 Balkus et al. 6,625,643 B1 9/2003 Colby et al. 2004/0268357 A1 12/2004 Joy et al. 6,658.477 B1 12/2003 Lisitsa et al. 2004/0268407 Al 12/2004 Sparrell et al. 6,684.331 B1 1/2004 Srivastava 2005.0005025 A1 1/2005 Harville et al. 6,687,664 B1 2/2004 Sussman et al. 2005, OO18775 A1 1/2005 Subramanian et al. 6,691.312 B 1 2/2004 Sen et al. 2005/0055517 A1 3, 2005 Olds et al. 6,694,368 B1 2/2004 An et al. 2005/0066082 A1 3, 2005 Forin et al. 6,711,171 B1 3, 2004 Dobbins et al. 2005/0081158 A1 4/2005 Hwang 6,725,274 B1 4/2004 Slik 2005, 0125734 A1 6/2005 Mohammed et al. 6,725,279 B1 4/2004 Richter et al. 2005, 0132168 A1 6, 2005 Weiss et al. 6,757,735 B2 6/2004 Apostolopulos et al. 2005/0172309 A1 8, 2005 Risan 6,802,019 B1 10/2004 Lauder 2005/01883 11 A1 8/2005 Diesel et al. 6,810,526 B1 10/2004 Menard et al. 2005, 01981.89 A1 9, 2005 Robinson et al. 25. R 1439 2005/0204289 A1 9, 2005 Mohammed et al. 6,957,430- 4 W B2 10/2005 FantOrter et al. 2005/0226324 A1 10/2005 Ouyang et al. 6.975,752 B2 12/2005 Dixon et all 2005, O262254 A1 11/2005 Sherwani 7,024.483- - B2 4/2006 Dinker et al. 2007.0011321 A1 1/2007 Huntington et al. 7,035,858 B2 4/2006 Dinker et al. 2008.OO37957 A1 2/2008 Nallur et al.

7,076,564 B2 7/2006 To et al. 7,124,424 B2 10/2006 Gordon et al. FOREIGN PATENT DOCUMENTS 7,139,925 B2 11/2006 Dinker et al. 7,197.535 B2 3/2007 Salesky et al. EP 0814403 (A1) 12/1997 7,206,854 B2 4/2007 Kauffman et al. JP 20O25.14797 5, 2002 7,240,325 B2 7, 2007 Keller WO WO962 1189 (A1) T 1996 7,246,318 B2 7/2007 Debique et al. WO WO9957837 A2 11/1999 7,290,057 B2 10/2007 Saunders et al. 7,299.485 B2 11/2007 Chaney et al. 7,330,542 B2 2/2008 Kauhanen et al. OTHER PUBLICATIONS 7,415,537 B1 8, 2008 Maes Cline, et al., “DirectShow RTP Support for Adaptivity in Networked 7,426,637 B2 9/2008 Risan et al. Multimedia Applications', IEEE, 1998, pp. 13-22. 2001/0000962 A1 5/2001 Rajan Dunn, et al., “The Design and Implementation of JalDiSM', pp. 1-38, 2001.0024455 A1 9, 2001 Thaler et al. Dec. 4, 2001. 2002/0051017 A1 5, 2002 Wishoff Olson, Robert, et al., “Remote Rendering Using Vitk and Vic.” 2002/0085581 A1 7/2002 Haucket al. website at http://www-unix.mcs.anl.gov/fl/publications/vis00 2002.0099842 A1 7/2002 Jennings vicvtk.pdf. 2 pages. US 7,664,882 B2 Page 3

Engel, Klaus et al., “Remote 3D Visualization Using Image-Stream Sullivan, et al., “Programming with the Java Media Framework”. ing Techniques' website at: http://citeseer.nj.nec.com/394248.html. Sean C. Sullivan, Loren Winzeler, Jeanine Deagen, and Deanna 4 pages. Brown, 1998, pp. 99. Engel, K., et al., “Combining Local and Remote Visualization Tech “How Terminal Services Works', retrieved on May 1, 2008 at niques for Interactive Volume Rendering in Medical Applications' <>, Microsoft pub/pub2000/engel vis00.pdf. TechNet, updated Mar. 28, 2003, pp. 1-15. TGS website at http://www.tgs.com/pro divoiv overview.htm. Maharaj, "A Primitive Window System Using GDI+ and C#”. OpenInventor from TGS4.0"Open Inventor Overview.” printed Apr. retrieved May 1, 2008 at html>, pp. 1-3. Journal of Information Sciences, vol. 107, pp. 85-105, 1998. Barth, et al., “Configuring Distributed Multimedia Applications Timothy K. Shih and Wen C. Pai, "A Stepwise Refinement Approach Using CINEMA', retrieved on Jan. 19, 2008 at < Published 2006, Arbab Ali Samejo et al., “GraphicalUser Interface Based Multimedia pp. 10. Web Suite in Windows Evironment'. Mehran Unversity Research Blome, et al., “Core Media Technology in Windows XP Empowers Journal of Engineering & Technology, vol. 20, No. 2, pp. 57-68, Apr. You to Create Custom Audio/Video Processing Components', 2001. retrieved on Sep. 22, 2009 at <>, MSDN Magazine, 16 pages. timedia Presentation', IEEE, pp. 528-532, 1997. Sei-Hoon Lee and Chang-Jong Wang, “Open Multimedia/ “DirectShow”, retrieved on Sep. 22, 2009 at >, Archived Oct. 17, 2002, 12 pages. Chi-Ming Chung, et al., “A Control and Data Abstraction Approach “EvCode.h', retrieved on Sep. 22, 2009 at , Microsoft, 2001, pp. 1-6. Robertson, Mark A.; Stevenson, Robert L.; "Temporal Resolution “GstEvent', retrieved on Sep. 22, 2009 at >, GStreamer Core Dame. Notre Dame, Indiana; Sep. 2001; pp. 1-11. Reference Manual, Archived Apr. 29, 2003, pp. 1-14. Girod, Bernd; Farber, Niko; "Feedback-Based Error control for “IFile:SourceFilter Interface”, retrieved on Sep. 22, 2009 at >, MSDN. Microsoft, Jul. 13, 2009, 4 pages. Chien, Shao-Yi; Chen, Ching-Yeh; Huang, Yu-Wen; and Chen, Taymans, et al., “GStreamer Application Development Manual”. Liang-Gee, “Multiple Sprites and Frame Skipping Techniques for retrieved on Sep. 22, 2009 at <>, Archived Apr. 5, 2003, pp. i-iv and 1-84. IEEE 2002: pp. 785-788. Thompson, “DirectShow for MediaPlayback in Windows', retrieved Agarwal, et al., “The Totem Multiple-Ring Ordering and Topology on Sep. 22, 2009 at <>, Parts 1-3, Last Part Dated Sep. 13, 200, 18 2003, pp. 129-140. pageS. Jaffe, et al., “Automatic Update of Replicated Topology Data Base'. ACM, 1984, pp. 142-148. * cited by examiner U.S. Patent Feb. 16, 2010 Sheet 1 of 4 US 7.664,882 B2

Computing System 100 Mass Storage

106

Main

Removable Storage

Memory 104 Non-Removable Storage

Processing w Volatile Unit Output Devices 102 110

input Devices

Non-Volatile 108

Network Connections 112 I

Remote Computing System 114

U.S. Patent Feb. 16, 2010 Sheet 2 of 4 US 7.664,882 B2

200 \

O Media Application 204 Byte 282 3. Stream Handlers Media Processing Module 202 216

Object Selection Module 214 28 Lookup Byte Stream Scheme Module Handler Handler 210 List(s) 215 List(s) 217

Scheme Pipeline 220 Handlers 218

Transform(s) Bit Pump Source 224 226 Object Media (media Content 2CCeSS Destination object) Transform(s) Bit Pump - Sink(s) 208 222

Media Content Store 206 Media Content 212 U.S. Patent Feb. 16, 2010 Sheet 3 of 4 US 7.664,882 B2

3OO Y. Determine URL Scheme 306

ldentify All Scheme Handlers Corresponding to URL Scheme 3O8

310 No

Scheme Handler to Try? Yes Try Next Scheme Handler 312

Scheme Handler Successful? 314

316 Yes Desired Object Type? SE NO

Use Byte Stream Handler? 318 Yes Identify All Byte Stream Handlers 320

322 Byte Stream Handler to Try? No Return Error

Y eS Try Next Byte Stream Handler 324

NO Byte Stream Handler Successful? 326

Yes N 328 O - Desired ObjectType? Yes U.S. Patent Feb. 16, 2010 Sheet 4 of 4 US 7.664,882 B2

677

US 7,664,882 B2 1. 2 SYSTEMAND METHOD FOR ACCESSING and requirements that can be used to present multimedia MULTIMEDIA CONTENT content. Individually programming the multimedia applica tions in this manner, however, is inefficient, particularly when RELATED APPLICATIONS a single computing system is configured with multiple appli cations that must each be programmed to interpret certain This application claims the benefit to U.S. Provisional basic information of the multimedia content. For instance, it application Ser. No. 60/546,460, filed Feb. 21, 2004. Addi may be necessary for the applications to each determine what tionally, this application is related to the following U.S. patent protocol to use to read the data, and once the data are being applications, each of which is hereby incorporated herein by read, the format of the data. reference, and each of which is assigned to the assignee of the 10 Programming each of the different applications to indi present application: U.S. patent application Ser. No. 10/730, vidually determine how the multimedia content should be 735, filed Dec. 8, 2003, to Mohammed et al., and titled read and what format it is requires duplicated efforts and “Media Processing Methods, Systems and Application Pro undesirably wastes time, costs, and resources that are gram Interfaces”; U.S. patent application Ser. No. 10/185. required to individually program the multimedia applica 161, to Kirt Debique, et al, filed Jun. 28, 2002, and titled 15 tions. Accordingly, there is currently a need in the art for Application Programming Interface for Utilizing Multime improved methods for enabling multimedia applications to dia Data”; U.S. patent application Ser. No. 10/735,522, to access and process multimedia data without the applications Weiss et al., filed Dec. 11, 2003, titled “Destination Applica having to individually interpret and determine the format of tion Program Interfaces”; U.S. patent application Ser. No. and the best method for reading the multimedia content 10/782,734, to Rahman, et al., filed Feb. 19, 2004, titled directly. “Media Timeline': and U.S. Provisional application Ser. No. 60/546,671, to Lipe, et. al., and titled “Function Discovery'. SUMMARY BACKGROUND Described herein are various systems and methods that 25 provide access to media content of varying types from a Users of computing devices, such as desktop PCs, set-top variety of sources using a uniform resource locator (URL). In boxes, personal digital assistants (PDAs), and so on, have accordance with one implementation, a first object operable access to an ever increasing amount of multimedia content to access data from a location specified by a URL is selected (e.g., audio, video, and other assorted data). Typically, this based on a scheme of the URL. A second object operable to multimedia content is accessed, manipulated and/or pre 30 read media content of a given type from the location specified sented on a computing device using a multimedia application by the URL is then selected based on data accessed using the running on the device. first object and/or file type information in the URL. Multimedia content may be accessed from a variety of different sources, in a variety of different formats, using a BRIEF DESCRIPTION OF THE DRAWINGS variety of different communications protocols. Programming 35 FIG. 1 is an illustration of one environment in which the each of the different applications to individually determine systems and methods described herein may be employed. basic information regarding the multimedia content requires FIG. 2 is a high level block diagram of one possible mul duplicated efforts and undesirably wastes time, costs, and timedia presentation system in which the various systems and resources that are required to individually program the mul methods described herein may be employed. timedia applications. In addition, most current applications 40 FIG. 3 illustrates one example of an operational flow do not provide an extensibility mechanism to allow for new including various operations for selecting a media Source types of Sources or formats. Accordingly, there is currently a component. need in the art for improved methods for enabling multimedia FIG. 4 illustrates another possible environment in which applications to access and process multimedia data without the systems and methods described herein may be employed. the applications having to individually interpret and deter 45 mine the characteristics of the multimedia content directly. DETAILED DESCRIPTION One underlying problem with using typical multimedia applications is that in order to access multimedia content, Described herein are various implementations of systems however, is that there are many different types of formats, and methods related to accessing multimedia content (re standards, and protocols that are used to encode and transmit 50 ferred to herein as “media content”). As will be appreciated digital data. Examples of different formatting standards by those skilled in the art, media content typically include include, for example, MPEG (Moving Picture Experts Some combination of audio, video, still or animated images, Group), ASF (Advanced Streaming Format), AVI (Audio information related to multiple display areas on a display Video Interleave), Active Movie, Cinepaq, QuickTime, screen, and other associated operational data. Media content Indeo, ATVEF (Advanced Television Enhancement Forum) 55 may be produced, streamed, and/or stored local (e.g., stored and ATSC (Advanced Television Systems Committee). locally on or in some form of computer-readable media), or Examples of different protocols and languages include produced, stored, or streamed from a remote location. RTSP (Real-Time Streaming Protocol), RTP (Real-Time Pro More particularly, described herein are various implemen tocol), RTCP (Real-Time Control Protocol), UDP (User tations of an object selection module. In general, an object Datagram Protocol), HTTP (Hypertext Transport Protocol), 60 selection module is used by a media system and/or media MMS (Microsoft Media Server), TCP (Transmission Control application to select an appropriate mechanism (e.g., source Protocol), XML (extensible Markup Language), and SMIL object or byte stream object) by which desired media content (Synchronized Multimedia Integration Language). may be accessed. As described fully below, the object selec Accordingly, the large variety of different multimedia for tion module makes these selection based on information con mats, standards, and protocols creates a burden on program 65 tained is a Uniform Resource Locator (URL) that specifies a mers for creating multimedia applications that are capable of location of the desired media content and/or file type infor accommodating the various standards, protocols, languages mation in the URL. US 7,664,882 B2 3 4 Before discussing the object selection module in detail, an cessing module 202 obtains and processes media content 212 example of a computing system 100 in which an object selec from the media content store 206 via a source object, pro tion module may be implemented will be described with cesses the media content 212 in Some manner, and outputs the respect to FIG.1. In its most basic configuration, the comput processed media content to the media content destination 208 ing system 100 includes a processing unit 102 and main via media sinks 228, inaccordance with directions or requests memory 104, including Volatile and/or non-volatile memory. from the media application 204. Additionally, the computing system 100 may include or have Turning now more particularly to the form and function of access to various mass storage devices or systems 106. the elements of the media processing environment, the media including various removable and/or non-removable mass application 204 may be any type of multimedia software storage devices. Examples of mass storage devices might be, 10 application that does playback, archiving/encoding, editing, without limitation, various magnetic, optical, and/or non or any other presentation or manipulation of multimedia con Volatile semiconductor memory, etc. In the case where the tent. In operation, the application 204 may indicate the loca mass storage device comprises a number of storage devices, tion of media content to be accessed, configure the media those devices may be distributed. Such as across a computer processing module 202 to processes the media content in a network. 15 particular manner, and instruct the media processing module The computing system 100 may have input devices 108, as to the media content destination 208 that specifies where Such as a keyboard, a pointing device (mouse), various optical the processed media content is to be sent and/or rendered. scanners or readers, microphones, video cameras, or various The media content store 206 may be any of a number of other computer input devices. The computing system 100 systems or mechanisms from which media content 202 may may also have output devices 110. Such as display devices, be accessed by the media processing module 202. For speakers, printers, or various other computer output devices. example, and without limitation, the media store 206 may be Other aspects of the computing system 100 may include Some form of computer-readable medium on a computer network or communications connections 112 to other system, either local or remote to the computer system includ devices, computers, networks, servers, etc., using either ing the media application. Media Store 206 may also refer to wired or wireless computer-readable media. For example, the 25 content that is live, i.e. in the process of being created during computing system 100 is shown in FIG. 1 as being connected the presentation. The media store 206 might also be an appli to a remote computing system 114. cation, system, or device, either local or remote, from which It should be appreciated that the remote computing system media content is streamed. While the precise form of the 114 may encompass various types computing systems or media content store 206 may vary, in accordance with the computing processes. For example, in one implementation, 30 various systems and methods described herein, there is some the remote computing system 114 is similar in basic structure mechanism by which media content may be located and and features to the computing system 100. Furthermore, the accessed using a URL, this mechanism may be specific to the computing system 100 and the remote computing system 114 media content store. may be a part of, or in communication with, computer net works, such as Wide Area Networks (WAN). Local Area 35 A media content destination 208 is an object or API that Network (LANs), the Internet, or any of various other com defines where media content is to be presented (e.g. a window, puter networks. disk file, and the like) and what happens to the media content. The computing system 100 illustrated in FIG. 1 is config In accordance with various implementations, the application ured as a personal computer (PC). However, the computing 204 itselfmay create the media content destination 208. Typi system 100 may also assume a variety of other configurations, 40 cally, the media content destination 208 will be associated Such as, without limitation, a mobile station, an entertainment with and receive processed media content from one or more appliance, a set-top box communicatively coupled to a dis media sinks (described below) in the media processing mod play device, a wireless phone, a video game console, a per ule 202. A more detailed description of media content desti Sonal digital assistant (PDA), and so forth. Thus, the comput nations may be had with reference to U.S. patent application ing system 100 may range from a full resource device with 45 titled “Destination Application Program Interfaces.” having Substantial memory and processor resources (e.g., PCs, tele Ser. No. 10/735,522, filed Dec. 11, 2003, incorporated herein vision recorders equipped with hard disk, etc.) to a low by reference. resource device with limited memory and/or processing The lookup module 210 is an application or process that is resources (e.g., a traditional set-top box). A more comprehen used by the object selection module 214, such as the media sively described example of a computing system 400 in which 50 processing module 202, to compile a list or lists 215 of byte the system and methods described herein may be imple stream handlers 216 and/or a list or lists 217 of scheme mented is shown in FIG. 4. handlers 218. (Byte stream handlers 216 and scheme handlers FIG. 2 illustrates one of many possible media processing 218 are described in detail below.) As used herein, the term environments 200 such as a component object-based multi “list describes any type of data structure or other mechanism media framework in which the storage, transmission, and/or 55 by which scheme handlers and byte stream handlers may be presentation of media content may take place. Likewise, FIG. associated or grouped, either directly or by reference. Gener 2 illustrates one of many possible media processing environ ally, the lookup module 210 uses information contained in a ments 200 in which various implementations of an object URL and/or information read from the location pointed to by selection module, may be employed, as will now be dis a URL in compiling lists. cussed. 60 The information from a URL that is used by the lookup Included in the media processing environment 200 is a module 210 may include information concerning or charac media processing module 202, a media application 204, a terizing the media content pointed to by the URL. Likewise, media content store 206, a media content destination 208, and information read from the location pointed to by the URL a lookup module 210, a number of byte stream handlers 216, may include information concerning or characterizing the and a number of scheme handlers 218. The details of each of 65 media content pointed to by the URL. This information is these elements of the media processing environment are then used by the lookup module 210 to create the lists of byte described below. However, in general terms, the media pro stream handlers 216 and/or scheme handlers 218. US 7,664,882 B2 5 6 The lookup module 210 may use various logic or mecha between, the various components being used by the media nisms to create the list or lists of byte stream handlers 216 processing module 202 in the processing and/or delivery of and/or scheme handlers 218. In some implementations, the media content. lookup module 210 uses the scheme of a URL, in conjunction In accordance with at least one implementation, the media with a lookup table, which associates Schemes with scheme processing module 202 comprises a media engine, which is a handlers, to select Scheme handlers that produce objects that functional component within an operating system of a com may be Suitable for accessing the media content pointed to by puting system. More particularly, in accordance with at least the URL. Likewise, the lookup module 210 may use other one implementation, the media processing module 202 com select information included in the URL and/or read from the prises a media engine in a media foundation platform, as location pointed to by the URL, in conjunction with a lookup 10 described in detail in U.S. patent application titled “Media table, which associates the information with byte stream han Processing Methods, Systems and Application Program dlers, to select byte stream handlers that produce objects that Interfaces,” having Ser. No. 10/730,735, filed Dec. 8, 2003, may be Suitable for accessing the media content pointed to by which is incorporated herein by reference. the URL. As shown, the media processing module 202 includes a In other implementations, the lookup module 210 may 15 number of components including an object selection module employ other mechanisms to create the lists of stream han 214 and a component pipeline 220. Such as a component dlers 216 and/or scheme handlers 218. For example, in accor object-based pipeline. As noted above, the object selection dance with one implementation, the lookup module 210 uses module 214 operates, in conjunction with the lookup module a “function discovery process or mechanism to create the 210, to select appropriate source objects for accessing media lists. A complete description of the operations of the function content. As shown in FIG. 2, the object selection module 214 discovery process are set forth in U.S. provisional application maintains one or more byte stream handler lists 215 and one Ser. No. 60/546,671, filed Feb. 20, 2004, to Lipe et al, and or more scheme handler lists 217. The operations of the object titled “Function Discovery', which is incorporated herein by selection module 214 are described in detail below. reference. Various functions of the lookup module 210 are Generally, the pipeline 220 includes various media content described in more detail below with respect to FIG.3. As used 25 processing objects, each of which perform some form of within the scope of this invention, function discovery serves media content processing or handling. The pipeline may as a store from which Scheme Handlers and Byte Stream include any number and/or type of media content processing Handlers can be enumerated and instantiated and from which objects, connected in various ways. However, with respect to various properties associated with those handlers can be the implementations described herein, the pipeline includes retrieved. 30 at least one source object for accessing media content. Details regarding the form and function of Source objects, and how In accordance with one implementation, scheme handlers they are selected for inclusion in the pipeline, are set forth and byte stream handlers comprise Component Object Model below. (COM) objects. As will be appreciated by those skilled in the Included in the example pipeline 220 shown in FIG. 2 are art, a COM is a component software architecture from 35 one or more source objects 222, one or more transform Microsoft Corporation which defines a structure for building objects 224, one or more of bit pumps 226, and one or more program routines or objects that can be called up and executed media sinks 228. Each component of the pipeline 220 pro in a Microsoft Windows(R) operating system environment. In vides some sort of processing or handling of media data. this implementation, Function discovery instantiates the Generally, the source object 222 accesses media content from scheme handlers and byte stream handlers via COM’s CoCre 40 a media content store, and reads and parses the media content atenstance function. into one or more streams of data. For example, in the particu Additionally, the lookup module 210 may include various lar implementation of a pipeline shown in FIG. 2, the source mechanisms by which an application or other process may object parses the media content data into two streams of data, register new and/or proprietary Scheme handlers and/or byte a video stream and an audio stream. stream handlers with the lookup module, such that these 45 Each stream of data may be handled or processed in some registered scheme handlers and byte stream handlers may be form by one or more transforms. As will be appreciated by selected based on information contained in the URL and/or those skilled in the art, there are various types of processing information accessed from the media content. In this regard, that may be carried out by such transforms with respect media in the case where the lookup module 210 employs the func content. As such, many different types of transforms may be tion discovery processes discussed above, the function dis 50 included in a transforms 224, and any number of (including covery provides just such a registration mechanism in the Zero) transforms may be applied to the stream. For example, form of APIs. The function discovery APIs provide a stan and without limitation, individual transforms may perform dardized mechanism by which third parties can register their Such functions as encoding, decoding, hue adjusting, contrast own scheme handlers and byte stream handlers, along with adjusting, equalization, frame rate adjusting, transition any associated properties, so that they get included in the 55 effects (wipes, fades, crossovers), Surround sound spatializa enumeration and will therefore be considered by object selec tion, and so on. tion module 214. Once the data in a stream has been processed by the trans In general, the media processing module 204 acts as a forms 224, the processed streams are passed via bit pumps central media content control and/or processing point for the 226 to corresponding media sinks 228. In general, media application. In at least one implementation, the media pro 60 sinks are responsible for sending the (possibly transformed) cessing module 202 exposes various components included multimedia data to wherever or whatever is specified by therein to the application 204 via application programming media application 204 in the media content destination 208. interfaces (APIs). The media processing module 202 may For example, in a playback application, the destination may also expose services via APIs by which the media application indicate a window, and the media sinks would be audio and 204 may configure these components. In this way, the media 65 video renderers that send the multimedia data to devices; in an processing system 202 abstracts away from the application encoding application, the destination may indicate a file to 204 the specific details of the existence of, and interactions which the encoded bits are to be archived, and the media sinks US 7,664,882 B2 7 8 would be responsible for arranging these bits in the appropri URL. More particularly, in accordance with various imple ate format and writing to the file, etc. mentations described herein, a source object may be one of A more detailed description of the form and function of two types, a simple source object that accesses one instance of pipelines and various pipeline components included in pipe media content or a timeline source object, described above, lines may be had with reference to U.S. patent application that accesses multiple instances of media content, which are titled “Media Processing Methods, Systems and Application to be invoked in a given temporal order. In at least one imple Program Interfaces.” having Ser. No. 10/730,735, filed Dec. mentation, the timeline source creates a source object for each 8, 2003, which is incorporated herein by reference. instance of content and uses those created source objects to As previously noted, the media processing module 202 access the media data. obtains and processes media content 212 using the Source 10 In general, a source object is a software module or API that object 222. More particularly, in a typical operational sce is operable to access media content of a particular format, nario, the application 204 requests the media processing mod from a given location, and in accordance with a particular ule 204 to provide access to and/or manipulation of particular access method, Such as aparticular communications protocol. media content from a particular media content store 206, to More particularly, a source object reads media content and transform the data, and to send the data to media sinks 228, as 15 produces output streams. In so doing, a source object per specified by destination 208. forms format-specific parsing to produce streams of data that Turning now more particularly to the manner in which are then sent through the other components in the pipeline Source objects are selected by the media processing module 220, and on to the media content destination 208. A more 202, in making a request for particular media content, the detailed explanation of source objects may be had with ref application 204 provides to the media processing module 202 erence to U.S. patent application Ser. No. 10/185,161, filed the location of the media content in the form of URL. Once Jun. 28, 2002, by Kirt Debique, et al., titled 'Application given the location (URL) of the media content 212, the media Programming Interface for Utilizing Multimedia Data.” processing module 202 then invokes the object selection which is incorporated herein by reference. module 214 to selector create, using the lookup module 210, Generally, a scheme handler 218 is a software module that an appropriate object (media access object 222) for accessing 25 takes as input a URL and is able to produce a byte stream (and potentially interpreting) the desired media content from object that will allow bits to be read from that URL. Compo the location specified by the URL. nents using the byte stream object output by a scheme handler The preceding paragraph describes just one of the ways in need not understand, in a programmatic sense, how to access which the object selection module may be invoked. To illus the data from wherever the URL is pointing or to communi trate that this can be generalized, consider the following 30 cate via the protocol that may be defined by that URL. Alter example. One example of a media source is the timeline natively, a scheme handler may be able to glean enough source. Timeline sources are described in detail in U.S. patent information from the URL to produce a source object. Typi application Ser. No. 10/782,734, titled “Media Timeline.” cally, a particular scheme handler will be tied to a specific incorporated herein by reference. Timeline sources are used medium and/or protocol, and the lookup module 210 will be to play metafiles like ASX, M3U, and other type of metafiles, 35 able to provide a list of available scheme handlers that will which contain a list of URLs specifying locations including among them be able to cover a wide variety of schemes. media content that is to be played in a certain order. For each Furthermore, a scheme handler can potentially also create a URL in the metafile, the timeline source uses the same object timeline object that can be used by a timeline media source. selection module 214 as described in the previous paragraph In general, a byte stream handler 216 is a software module to create an underlying source. The application 204 and the 40 that takes as its input a byte stream and produces as an output media processing module 202 are unaware, in a program a source object that is capable of understanding the bits in a matic sense, of this process. There may be other modules that byte stream. Typically, a particular byte stream handler will implement the same interface as the object selection module be tied to a specific format and will be able to identify that the 202. The application 204 or other objects may use these bits in the byte stream are for its format; if so, it will create the special implementations for certain dedicated scenarios. 45 appropriate source object, in the form of media source object Before described the manner or manners in which the 222. The lookup module 210 will be able to provide a list of object selection module 214 selects or invokes an appropriate available byte stream handlers that will, among themselves, object, it will be helpful to first to describe in more detail the be able to understanda wide variety of formats. Furthermore, various handlers and an objects that may be selected or used a byte stream handler can potentially also create a timeline by the object selection module 214, as well as discussing the 50 object that can be used by a timeline media Source. form and format of URLS. Typically, a byte stream handler will attempt to validate In general, a byte stream object is a software module or API that the given byte stream is in the appropriate format for that includes appropriate logic and/or data for accessing parsing by the source object that it will create. The process by media content from a given location and in accordance with a which this is done is format specific, but normally involves particular access method, Such as a particular communica 55 reading a certain amount of the byte stream and ensuring its tions protocol. More particularly, a byte stream object is format specific validity. pointed to a particular location/medium from which data are As will be appreciated by those skilled in the art, a URL is available and produces as output a stream of bits that can be a standardized address for a resource. URLs are used to read by a component using the byte stream. A byte stream “locate' resources, by providing an abstract identification of object is a useful abstraction that provides a component using 60 the resource location. Having located a resource, a system them with a way of reading data from a medium, possibly may perform a variety of operations on the resource, as might over a certain protocol, without that component understand be characterized by such words as “access.” “update.” ing, in a programmatic sense, the particulars of how to obtain “replace.” etc. In general, only the “access' method needs to data from that medium. be specified for any URL scheme. A source object may be any of a variety of objects that 65 In general, URLs are written as follows: . include appropriate logic and/or data to access the specific telnet”, “wais', and so on. Additionally, to the scheme handlers associated with the scheme identified in the schemes specified by the IANA, individuals or organizations 10 determine URL scheme operation 306. may create and use their own schemes. Of course, for these In accordance with another implementation, the identify schemes to be used the application or process using the scheme handler operation 308 uses a “function discovery' schemes must be aware of the form of the schemes and the process to identify the appropriate scheme handlers. As pre access methods that are associated therewith. viously described, the operations of the function discovery As with the scheme, the scheme dependent information has 15 process are set forth in U.S. Provisional application Ser. No. a specified form. The form of the scheme dependent informa 60/546,671, filed Feb. 20, 2004, to Lipe. et al., and titled tion may vary, but typically the scheme dependent informa “Function Discovery'. tion includes an address for the machine making the resource In accordance with various implementations, the schemes available and the “path' to that resource. Typically, the handlers identified in the identify scheme handler operation scheme is separated from the machine address by two slashes 308 may be compiled in an ordered data structure, such as an (//), and the machine address is separated from the path with ordered list. This list may contain references to one or more one slash (/). The path may further include information indi scheme handlers. The list may also include no scheme han cating various directories and Sub-directories where the dlers. resource is located, the name of the resource, and, in most Next, a scheme handler determination operation 310 deter cases, the type of the resource, as indicated by an extension to 25 mines if the identify scheme handler operation 308 was able the resource name (e.g., resource.extension). to find at least one appropriate scheme handler. Again, this Turning now to FIG.3, shown therein is an operational flow may be done in a number of ways. For example, in imple 300 including various operations that may be performed by mentations where the identify scheme handler operation 308 the object selection module 214 in selecting or instantiating created a data structure, the scheme handler determination an appropriate Source object to access media content for a 30 operation 310 may simply determine whether there are any given URL. In accordance with various implementations, scheme handlers in the data structure. when the application requests that specific media content be If the scheme handler determination operation 310 deter played, the application will pass the media processing mod mines that there are no scheme handlers to try, an error value ule, among other things, the URL of the media content. The is returned to the caller and the operational flow 300 ends. If, media processing module 202 will invoke the media object 35 however, the scheme handler determination operation 310 selection module 214 and pass the URL and the object type to determines that there is at least one scheme handler to try, a try the object selection module 214; in this case, the object type scheme handler operation 312 then tries one of the available desired is a Media Source. In accordance with various imple scheme handlers. In the case where the scheme handlers are mentations, the object selection module 214 will then per listed in an ordered data structure, the next untried scheme form the operations illustrated in the operational flow 300, as 40 handler will be tried by the try scheme handler operation 312. will now be described. If no untried scheme handlers are available to be tried in list, In accordance with some implementations, the operations then an error value is returned to the caller in the operational flow 300 will be performed entirely by the In accordance with various implementations, trying a object selection module 214. In other implementations, the scheme handler involves invoking the scheme handler. As a operations in the operational flow 300 will be performed by 45 result of invoking the scheme handler, one of three things may the object selection module in conjunction with the lookup happen: a byte stream object may be produced, a source module 210. For example, in accordance with one implemen object may be produced, a timeline object may be produced, tation, the lookup module will perform the identify operations or no object will be produced. The scheme handler is invoked 308 and/or 320 of the operations flow 300. In yet other imple by using the scheme handler API/interface. mentations, the operations of the operational flow 300 may be 50 Next, a success determination operation 314 determines performed by other processes or modules either within or in whether the scheme handler was successful. That is, it is conjunction with the media processing module, or by some determined if the scheme handler tried in operation 312 pro other process or module altogether distinct from the media duced eitherabyte stream object, a source object or a timeline processing module 202, the object selection module 214, object. If the scheme handler tried in operation 312 did not and/or the lookup module 210. 55 produce either a byte stream object or a source object, the As shown, at the beginning of the operational flow 300, a operational flow returns to the scheme handler determination URL scheme determination 306 determines the scheme of the operation 310. However, if the scheme handler tried in opera URL which points to the desired media content. Following tion 312 did produce either a byte stream object, a source the URL scheme determination operation 306, an identify object, or a timeline object, a determine desired type opera scheme handlers operation 308 then uses the determined 60 tion 314 determines whether the type of object produced was scheme to identify all available scheme handlers that are the type of object requested by the application. associated with the scheme of the URL. That is, all of the If the determine desired type operation 314 determines that available scheme handlers that have been identified in some the type of object produced was the type of object requested manner as being operable to read information in accordance by the application, the object is returned. If, however, the with the identified scheme. 65 determine desired type operation 314 determines that the type The manner in which the identify scheme handler opera of object produced was not the type of object requested by the tion 308 identifies the appropriate scheme handlers may be application (typically, in this case, it is a byte stream that was US 7,664,882 B2 11 12 produced), a byte stream handler determination operation in a single list based on the cost of the byte stream handlers. 318 then determines if a byte stream handler should be used. For example, in one implementation the byte stream handlers If the produced object is a byte stream and the requested that area associated with the extension identified in the URL object is a media source or timeline object, then abyte stream and/or the MIME type of the byte stream are ordered such that handler can be used to create the requested object the following try byte stream handler operation 324 selects If the byte stream handler determination operation 318 the untried byte stream handler having the lowest cost. determines that a byte stream handler is not to be used the In accordance with another implementation, the byte operational flow 300 returns to the scheme handler determi stream handlers identified as being associated with the exten nation operation 310. However, if the byte stream handler sion identified in the URL and/or the MIME type of the byte determination operation 318 determines that a byte stream 10 stream may be compiled in an ordered data structure accord handler may be used, the operational flow proceeds to an ing to cost, such as an first ordered list, while all other byte identify all byte stream handler operation 320. stream handlers are included in a second list. In this imple The identify byte stream handler operation 320 uses vari mentation, the following try byte stream handler operation ous information derived from the URL, and in some cases 324 would first try the untried byte stream handler having the other information related to or present in the byte stream, to 15 lowest cost from the first list. In the case where all byte stream identify byte stream handlers that are associated with or iden handlers in the first list were already tried, or where there tified by the information derived from the URL. For example, never were byte stream handlers in the first list, the try byte and without limitation, in accordance with one implementa stream handler operation 324 would try an untried byte tion, the identify byte stream handler operation 320 uses the stream handler in the second list. In accordance with one extension of the URL and/or the Multi-Purpose Internet Mail implementation, the byte stream handlers in the second list Extensions (MIME) information in the byte stream to identify may also be ordered according to their respective costs. appropriate byte stream handlers. In accordance with one implementation, the caller of object The manner in which the identify byte stream handler selection module 314 may have specified a source resolution operation 320 identifies these byte stream handlers may be option indicating whether or not this second list is to be accomplished in a number of ways. For example, in accor 25 created and used. For example, electing for this option would dance with one implementation, the information derived from ensure successful resolution if a WMV file is misnamed and the URL is the extension portion of the URL, if available. The given the incorrect extension “.mp3. In this case, the MP3 identify byte stream handler operation 320 then examines a byte stream handler will not succeed on this content, since it byte stream handler data structure including a number of is not in the MP3 format, but if the source resolver is allowed extensions and/or MIME types and a number of byte stream 30 to try other byte stream handlers, the WMV byte stream handlers. In accordance with this implementation, the byte handler will be found on the second list and will succeed on stream handler data structure associates each unique exten this content. However, trying the byte stream handlers on the sion and/or MIME type with all of the available byte stream second list is more costly than trying only those on the first handlers associated with that extension and/or MIME type. list, and therefore some callers may choose not to use the The identify byte stream handler operation320 then compiles 35 second list. alist or lists, or the like, of the stream handlers associated with Following the identify byte stream handler operation 320, the extension identified in determine URL. a byte stream handler determination operation 322 deter In accordance with another implementation, the identify mines if the identify byte stream handler operation 320 was byte stream handler operation 320 uses a “function discov able to find at least one appropriate byte stream handler. ery' process to identify the appropriate byte stream handlers. 40 Again, this may be done in a number of ways. For example, in As previously described, the operations of the function dis implementations where the identify byte stream handler covery process are set forth in U.S. provisional application operation320 created a data structure, the byte stream handler Ser. No. 60/546,671, filed Feb. 20, 2004, to Lipe et al., and determination operation 322 may simply determine whether titled “Function Discovery. there are any byte stream handlers in the data structure. In accordance with various implementations, the byte 45 If the byte stream handler determination operation 322 stream handlers identified as being associated with the exten determines that there are no byte stream handlers to try, an sion identified in the URL and/or the MIME type of the byte error value is returned to the user and the operational flow 300 stream may be compiled in an ordered data structure. Such as ends. If, however, the byte stream handler determination an ordered list. This list may contain references to one or more operation 322 determines that there is at least one byte stream byte stream handlers. The list may also include nobyte stream 50 handler to try, a try byte stream handler operation 324 then handlers. tries one of the available byte stream handlers. In the case The order of the byte stream handlers in the data structure where the byte stream handlers are listed in an ordered data in this embodiment may be selected in a number of ways. For structure, the next untried byte stream handler will be tried by example, in accordance with one implementation, each of the the try byte stream handler operation 324. byte stream handlers will have associated therewith a prede 55 In accordance with various implementations, trying a byte termined cost value, which is used in the identify byte stream stream handler involves invoking the byte stream handler on handler operation 320 in ordering the byte stream handlers. the byte stream via the byte stream handler API/Interface. The cost value may be determined in various ways and may Next, a success determination operation 316 determines be indicative of various properties of the byte stream handler whether the byte stream handler was successful. That is, it is to which it is associated. For example, in accordance with one 60 determined if the byte stream handler tried in operation 324 implementation, the cost value is related to the amount of data produced the object desired by the caller. If the byte stream that must be read from the byte stream before the following handler tried in operation 324 produced an object, a deter byte stream handler determination operation 326 can deter mine desired type operation 328 determines whether the type mine if the byte stream handler was successful. of object produced was the type of object requested by the In accordance with one implementation, the byte stream 65 application. handlers that area associated with the extension identified in If the determine desired type operation 328 determines that the URL and/or the MIME type of the byte stream are ordered the type of object produced was the type of object requested US 7,664,882 B2 13 14 by the application, the object is returned. If, however, the contemplated that the media processing module 202, the des determine desired type operation 328 determines that the type tination 208, the object selection module 214, the byte stream of object produced was not the type of object requested by the handlers 216, the scheme handlers 218, the components of the application, the operational flow 300 proceeds back to the pipeline 220, and the lookup module 210 and their operations byte stream handler determination operation 322. and functions, such as the operations shown and described In accordance with one implementation, the caller of the with respect to 3 and 4, may be implemented as hardware, object selection module 214 may already have a byte stream Software, firmware, or various combinations of hardware, object (instead of an URL). In this case, it requests that object Software, and firmware. The implementation is a matter of selection module 214 create the desired object from a byte 10 choice. stream. In such a case, the operational flow 300 would simply The previous discussion presented a system level overview begin at the identify byte stream handler operation 320. of modules and components that are used in the process of In accordance with one implementation, if this process accessing media content. The following discussion relates to fails while attempting to find an appropriate byte stream novel APIs and data structures related to the modules and handler, the caller of object selection module 314 can request 15 that the byte stream object that was created during the process components described above. The APIs detailed below are be kept alive (cached). The caller may request this option if it described in terms of the C++ programming language. How wishes to subsequently try to resolve the failed URL with the ever, the APIs are not limited the C++ programming lan “content does not have to match extension' option; this guage. Rather, the APIs may be defined and implemented in allows the second-chance resolution to take place without a any programming language, as those of skill in the art will second attempt to read from the URL, which may be costly. recognize. The names given to the API functions and param Generally, the media processing module 202, the object eters which will now be described are meant to be descriptive selection module 214, the byte stream handlers 216, the of their function. However, other names or identifiers could scheme handlers 218, the components of the pipeline 220, the be associated with the functions and parameters, as will be destination 208, and the lookup module 210 may be com 25 apparent to those of skill in the art. posed of or include computer-executable instructions, Vari In accordance with one particular implementation, the ous routines, programs, objects, components, data structures, etc., that perform particular tasks or operations and/or imple APIs are implemented with respect to the Microsoft(R) Com ment particular abstract data types, achorall of which may be ponent Object Model (COM). However, the APIs may like embodied in or on a computer readable medium. For 30 wise be implemented in a number of other ways and/or in example, in various implementations the object selection accordance with other models. As those skilled in the art will module 214 performs the operations illustrated in FIGS. 3 appreciate, the Microsoft(R) Component Object Model and/or 4. (COM) is a platform-independent, distributed, object-ori A computer-readable medium may include, without limi ented system for creating binary Software components that 35 can interact. COM is the foundation technology for tation, both volatile and nonvolatile memory, mass storage Microsoft's OLE (compound documents), ActiveX (Internet devices, and removable and non-removable media. enabled components), as well as others. COMallows objects Any or all of the media processing module 202, the desti to interact across process and machine boundaries as easily as nation 208, the object selection module 214, the byte stream within a single process by specifying that the only way to handlers 216, the scheme handlers 218, the components of the 40 pipeline 220, and the lookup module 210 may be executed or manipulate the data associated with an object is through an implemented in a single computing device. Alternatively, any interface on the object. or all of the media processing module 202, the destination As used with respect to the COM implemented interfaces 208, the object selection module 214, the byte stream han described herein, the term “interface” refers to an implemen dlers 216, the scheme handlers 218, the components of the 45 tation in code of a COM binary-compliant interface that is pipeline 220, and the lookup module 210 may be executed or associated with an object. A COM interface refers to a pre implemented in a distributed computing environment, where defined group of related functions that a COM class imple various operations are performed by remote processing ments, but a specific interface does not necessarily represent devices or systems that are linked through a communications all the functions that the class Supports Referring to an object network. For example, in accordance with one embodiment, 50 implementing an interface means that the object uses code the media processing module 202 is executed or implemented that implements each method of the interface and provides in one computing system, while the media content store 206 COM binary-compliant pointers to those functions to the is implemented or present in another computing system or COM library. COM then makes those functions available to device. any client who asks for a pointer to the interface, whether the It should be understood that while the media processing 55 clientis inside or outside of the process that implements those module 202, the destination 208, the object selection module functions. The various APIs set forth herein are described in 214, the byte stream handlers 216, the scheme handlers 218, a manner that is generally used with respect to the Microsoft(R) the components of the pipeline 220, and the lookup module COM, as will be appreciated and understood by those skilled 210 are described herein as comprising computer executable in the art. instructions embodied in computer-readable media, media 60 processing module 202, the destination 208, the object selec As will be appreciated by those skilled in the art. COM tion module 214, the byte stream handlers 216, the scheme interfaces typically include or inherit various methods and/or handlers 218, the components of the pipeline 220, and the definitions from IUnknown. The IUnknown interface lets lookup module 210, and any or all of the functions or opera clients get pointers to other interfaces on a given object tions performed thereby, may likewise be embodied all or in 65 through the IUnknown:QueryInterface method, and manage part as interconnected machine logic circuits or circuit mod the existence of the object through the IUnknown::AddRef ules within a computing device. Stated another way, it is and IUnknown::Release methods. Each of the IUnknown: US 7,664,882 B2 15 16 QueryInterface. The IUnknown::AddRef and IUnknown: Release methods are defined as follows: -continued can also be called when the life of a cloned pointer must extend beyond the lifetime of the original pointer. The cloned pointer must be released by calling IUnknown::Release. Unknown::Release Objects must be able to maintain (231)-1 outstanding pointer Release decrements the reference count for the calling interface on a object. If the reference count on the object falls to 0, the object is references. Therefore, the internal reference counter that reed from memory. IUnknown::AddRef maintains must be a 32-bit unsigned integer. ULONG Release(void); Return Value 10 Returns the resulting value of the reference count, which is used for In accordance with one implementation, in addition to the diagnostic/testing purposes only. IUnkown interface, the object selection module 214 imple Remarks ments one API, a source resolver interface (“IMFSourceRe fIUnknown::AddRefhas been called on this object's interface in solver) 280 and is capable of using a caller supplied resolu imes and this is the n + 1th call to IUnknown::Release, the implementation of IUnknown::AddRefmust cause the interface tion callback interface (“IMFResolutionCallback”) 282, as pointer to free itself. When the released pointer is the only existing 15 shown in FIG. 2. The IMFSourceResolver interface 280 reference to an object (whether the object Supports single or multiple exposes seven methods, which are defined as follows: interfaces), the implementation must free the object. Unknown: QueryInterface QueryInterface returns a pointer to a specified interface on an object o which a client currently holds an interface pointer. This function IMFSourceResolver: CreateCbjectFrom URL calls IUnknown::AddRef on the pointer it returns. The Create0bjectFrom URL method provides a synchronous way of HRESULT QueryInterface( creating a resolution object from the specified URL. REFIID id, Syntax void **ppvObiect HRESULT CreateCobjectFromURL( ); in LPCWSTR pwszURL, Parameters in DWORD dwFlags, iid 25 in IProperty Store pProps, in Identifier of the interface being requested. out MFOBJECT TYPE *pCbjectType, ppvObject out IUnknown **ppObject out Address of pointer variable that receives the interface Parameters pointer requested in rid. Upon Successful return, *ppvObject pwSZURL contains the requested interface pointer to the object. If the in Pointer to URL to be resolved. object does not support the interface specified in iid, 30 dwFlags *ppvObject is set to NULL. in Specifies resolution flags. See resolution flags section. Return Value pProps S OK if the interface is supported E NOINTERFACE if not. in Holds the optional property store object that gets passed Remarks to the underlying scheme and byte stream handlers. It can The QueryInterface method gives a client access to other interfaces on an object. also contain source resolver flags under the property key of 35 MFPKEY Resolution Flags. If a property store it not to be For any one object, a specific query for the IUnknown interface on used, then this parameter should be set to NULL. any of the object's interfaces returns the same pointer value. This pObjectType allows a client to determine whether two pointers point to the same out Specifies type of object that is being returned. Possible component by calling QueryInterface on both and comparing the values are: results. It is specifically not the case that queries for interfaces (even MF OBJECT MEDIASOURCE he same interface through the same pointer) must return the same pointer value. 40 MF OBJECT BYTESTREAM MF OBJECT PLAYLIST There are four requirements for implementations of QueryInterface MF OBJECT TIMELINE (In these cases, must succeed means “must Succeed barring MF OBJECT INVALID catastrophic failure.): ppObject First, the set of interfaces accessible on an object through out Holds the object that is being returned. Unknown::QueryInterface should be static, not dynamic. This Return Values means that if a call to QueryInterface for a pointer to a specified 45 If the method succeeds, it returns S OK. If it fails, it returns an error interface Succeeds the first time, it must Succeed again, and if it fails code. he first time, it must fail on all Subsequent queries. Remarks Second, the interface must be reflexive—if a client holds a pointer None. o an interface on an object, and queries for that interface, the call IMFSourceResolver:Create0bjectFrom ByteStream must Succeed. The Create0bjectFrom ByteStream method provides a synchronous Third, the interface must be symmetric—if a client holding a 50 way of creating a resolution object for the specified byte stream. pointer to one interface quenes successfully for another, a query Syntax hrough the obtained pointer for the first interface must succeed. HRESULT CreateCobjectFrom ByteStream( Fourth, the interface must be transitive—ifa client holding a in IMFByteStream pByteStream, pointer to one interface queries successfully for a second, and in LPCWSTR pwszURL, hrough that pointer queries successfully for a third interface, a query in DWORD dwFlags, or the first interface through the pointer for the third interface must 55 in IProperty Store pProps, Succeed. out MF OBJECT TYPE *pCbjectType, Onknown::AddRef out IUnknown **ppObject Onknown::AddRef increments the reference count for an interface ) on an object. It should be called for every new copy of a pointer to Parameters an interface on a given object. pByteStream ULONG AddRefvoid); in Pointer to a byte stream object. Return Value 60 pwSZURL Returns an integer from 1 to n, the value of the new reference count. in Optional pointer to the URL that matches the byte This information is meant to be used for diagnostic? testing purposes stream. If no URL is supplied, then the only, because, in certain situations, the value may be unstable. MF RESOLUTION CONTENT DOES Remarks NOT HAVE TO Objects use a reference counting mechanism to ensure that the MATCH EXTENSION resolution flag may be specified so ifetime of the object includes the lifetime of references to it. You 65 the source resolver will enumerate through the registered byte use IUnknown::AddRefto stabilize a copy of an interface pointer. It stream handlers to Successfully create a resolution object. US 7,664,882 B2 17 18

-continued -continued dwFlags ppObject in Specifies resolution flags. See resolution flags section. out Holds the resolution object that is being returned. pProps 5 Return Values in Holds the optional property store object that gets passed If the method succeeds, it returns S OK. If it fails, it returns an error to the underlying scheme and byte stream handlers. It can code. also contain source resolver flags under the property key of IMFSourceResolver::BeginCreate0bjectFrom ByteStream MFPKEY Resolution Flags. If a property store it not to be The BeginCreateCobjectFrom ByteStream method provides an used, then this parameter should be set to NULL. asynchronous way for creating a resolution object that for the pObjectType 10 specified byte stream. out Specifies type of object that is being returned. Possible Syntax values are: BeginCreate0bjectFrom ByteStream( MF OBJECT MEDIASOURCE in IMFByteStream pByteStream, MF OBJECT BYTESTREAM in LPCWSTR pwszURL, MF OBJECT PLAYLIST in DWORD dwFlags, MF OBJECT TIMELINE 15 in IProperty Store pProps, MF OBJECT INVALID out IUnknown * *ppIUnknownCancelCookie ppObject in IMFAsyncCallback *pCallback, out Holds the resolution object that is being returned. inl IUnknown punkState Return Values If the method succeeds, it returns S OK. If it fails, it returns an error Parameters code. pByteStream IMFSourceResolver::BeginCreate0bjectFrom URL in Pointer to a byte stream object. The BeginCobjectSourceFromURL method provides an asynchronous pwSZURL way for creating a resolution object for the specified URL. in Optional pointer to the URL that matches the byte Syntax stream. If no URL is supplied, then the HRESULT BeginCreateCbjectFromURL( MF RESOLUTION CONTENT DOES NOT HAVE in LPCWSTR pwszURL, TO MATCH EXTENSION resolution flag will preferably be in DWORD dwFlags, 25 specified so the source resolver will enumerate through the in IProperty Store pProps, registered byte stream handlers to Successfully create a out IUnknown * *ppIUnknownCancelCookie resolution object. in IMFAsyncCallback *pCallback, dwFlags in IUnknown punkState in Specifies resolution flags. See resolution flags section. ); pProps Parameters 30 in Holds the optional property store object that gets passed pwSZURL to the underlying scheme and byte stream handlers. It can in Specifies the URL to be resolved. also contain source resolver flags under the property key of dwFlags MFPKEY Resolution Flags. If a property store it not to be in Specifies resolution flags. See resolution flags section. used, then this parameter should be set to NULL. pProps pplUnknownCancelCookie in Holds the optional property store object that gets passed 35 out Holds the cancellation cookie that can be used in a call to the underlying scheme and byte stream handlers. It can to CancelObjectCreation(...) to abort the resolution process. also contain source resolver flags under the property key of This parameter is optional and can be NULL. It is the MFPKEY Resolution Flags. If a property store it not to be responsibility of the caller to Release the IUnknown pointer. used, then this parameter should be set to NULL. pCallback ppIUnknownCancelCookie in Pointer to an async callback object out Holds the cancellation cookie that can be used in a call punkState to CancelObjectCreation(...) to abort the resolution process. 40 in Pointer to a context object associated with this call This parameter is optional and can be NULL. It is the Return Values responsibility of the caller to Release the IUnknown pointer. f the method succeeds, it returns S OK. If it fails, it returns an error pCallback code. in Pointer to an async callback object. MFSourceResolver:EndCreateCobjectFrom ByteStream punkState The EndCreate0bjectFrom Byte Stream method completes an in Pointer to a caller defined context object associated with 45 asynchronous operation started with the this call. BeginCreateCobjectFrom ByteStream call and returns the resolution Return Values object if the operation Succeeds. If the method succeeds, it returns S OK. If it fails, it returns an error Syntax code. HRESULT EndCreateCbjectFrom ByteStream( IMFSourceResolver:EndCreateCbjectFrom URL in IMFAsyncResult *pResult, The EndCreate0bjectFrom URL method completes the asynchronous 50 out MF OBJECT TYPE *pCbjectType, operation started with the BeginCreateCobjectFrom URL call and out IUnknown **ppObject returns the resolution object if the operation Succeeds. Parameters Syntax pResult HRESULT EndCreateCbjectFromURL( in Pointer to an async result object passed in through in IMFAsyncResult *pResult, Invoke. out MFOBJECT TYPE *pCbjectType, 55 pObjectType out IUnknown **ppObject); out Specifies type of object that is being returned. Possible Parameters values are: pResult MF OBJECT MEDIASOURCE in Pointer to an async result object passed in through MF OBJECT BYTESTREAM Invoke. MF OBJECT PLAYLIST pObjectType MF OBJECT TIMELINE out Specifies type of object that is being returned. Possible 60 MF OBJECT INVALID values are: ppObject MF OBJECT MEDIASOURCE out Holds the resolution object that is being returned. MF OBJECT BYTESTREAM Return Values MF OBJECT PLAYLIST If the method succeeds, it returns S OK. If it fails, it returns an error MF OBJECT TIMELINE code. MF OBJECT INVALID 65 IMFSourceResolver:CancelObjectCreation The CancelObjectCreation method attempts to cancel the requested US 7,664,882 B2 19 20

-continued -continued resolution associated with the provided cancel cookie. The cancel Parameters cookie was obtained from one of the asynchronous pwSZURL BeginCreate0bjectFromXXX calls. Note that canceling an existing in The URL that is being resolved. resolution request is deemed to be “best-effort and might not truly pfContinueResolutionProcess yield the expected resource relinquishing effect as the resolution in Pointer to a flag that the indicates whether to continue might have already completed and the user callback not invoked yet. with resolution. Syntax Return Values HRESULT CancelObjectCreation.( If the method succeeds, it returns S OK. If it fails, it returns an error in IUnknown *pIUnknownCancelCookie 10 code. IMFReSolutionCallback:HandleNoMatchingExtensionAndDataFound Parameters The HandleNoMatchingExtensionAnd DataFound method is called pIUnknownCancelCookie by the object selection module 214 if no appropriate scheme in Pointer to an IUnknown object that was provided by a handler is found during the resolution process. call to either BeginCreate0bjectFrom URL or Syntax BeginCreate0bjectFrom ByteStream. 15 HRESULT HandleNoMatchingSchemeResolverFound ( Return Values in LPCWSTR pwszURL, If the method succeeds, it returns S OK. If it fails, it returns an error out BOOL* pfrontinueResolutionProcess code. ); Parameters pwSZURL in The URL that is being resolved. As previously noted, in accordance with one implementa pByteStream tion, in addition to the IUnkown interface, the object selection in Pointer to a byte stream object. module 214 implements one API, a source resolver interface pfContinueResolutionProcess in Pointer to a flag that the indicates whether to continue (“IMFSourceResolver) 280 and is capable of using a caller with resolution by enumerating through the byte stream supplied resolution callback interface (“IMFResolutionCall handlers. back') 282. IN accordance with one implementation, the 25 Return Values object selection module 214 allows a resolution callback to be If the method succeeds, it returns S OK. If it fails, it returns an error specified in the property store of the BeginCreationObject code. From URL and BeginCreateCobjectFrom ByteStream meth ods. If an IMFResolutionCallback object is provided under As previously noted, the scheme handler 217 has the the MFPKEY Resolution Callback property key as an IUn 30 responsibility to connect and communicate with a resource known, the MF Source Resolver will use the caller supplied for the scheme (i.e., a protocol) it is registered to handle. callback interface during resolution to allow the caller to Upon completion of the process, the scheme handler 217 will assist in resolution. This is convenient when an application either return a media source that can handle the underlying has its own scheme and/or bytestream handler that is wishes 35 data, return a byte stream object, return a timeline object, or to use, but it does not want to register it with Media Founda fail with an error. In accordance with one implementation, tion so that other applications cannot use it. these tasks are facilitated using, in addition to the IUnknown The IMFResolutionCallback interface 282 exposes three interface, an IMFSchemeHandler interface 286. The IMF methods, which are defined as follows: SchemeHandler interface 286 exposes three methods, which 40 are defined as follows:

IMFResolutionCallback::HandleResolution The HandleResolution method is called by the object selection IMFSchemeHandler:BeginCreateObject module 214 before resolution begins to give the caller a chance to The BeginCreateCobject method provides an asynchronous way for specify the byte stream handler that should be tried first. 45 creating an object that can connect and read data from the resource Syntax pointed to by the specified URL. HRESULT HandleResolution.( Syntax in LPCWSTR pwSZURL, HRESULT BeginCreateCbject( in IMFByteStream * pByteStream, in LPCWSTR pwszURL, out IMFByteStreamHandler **ppIMFByteStreamHandler in DWORD dwFlags, ); 50 in IProperty Store pProps, Parameters out IUnknown **ppIUnknownCancelCookie pwSZURL in IMFAsyncCallback *pCallback, in The URL that is being asked to resolve. inl IUnknown punkState pByteStream Parameters in Pointer to a byte stream object. pwSZURL ppIMFByteStreamHandler 55 in Pointer to URL to be resolved. out Pointer to a byte stream object that the Source Resolver dwFlags should use first in the resolution process. in Specifies resolution flags. See resolution flags section. Return Values pProps If the method succeeds, it returns S OK. If it fails, it returns an error in Holds the optional property store object that can contain code. other scheme resolution related objects and flags. If a IMFResolutionCallback::HandleNoMatchingSchemeResolverFound property store it not to be used, then this arg should be set to The HandleNoMatchingSchemeResolverFound method is called by 60 NULL. the object selection module 214 if no appropriate scheme handler is ppIUnknownCancelCookie found during the resolution process. out Holds the cancellation cookie that can be used in a call Syntax o CancelObjectCreation(...) to abort the resolution process. HRESULT HandleNoMatchingSchemeResolverFound.( This parameter is optional and can be NULL. It is the in LPCWSTR pwszURL, responsibility of the caller to Release the IUnknown pointer. out BOOL* pfContinueResolutionProcess 65 pCallback in Pointer to an async callback object US 7,664,882 B2 21 22

-continued -continued punkState in DWORD dwFlags, in Pointer to a context object associated with this call in IPropertyStore *pProps, Return Values out IUnknown * *ppIUnknownCancelCookie If the method succeeds, it returns S OK. If it fails, it returns an error in IMFAsyncCallback *pCallback, code. in IUnknown punkState IMFSchemeHandler:EndCreateObject ); The EndCreate0bject method completes the asynchronous call Parameters started in BeginCreateCobject and returns the appropriate object that pByteStream can connect and read data from the specified URL. Upon Successful 10 in Pointer to a byte stream object. completion of this call, the caller can verify the value of pwSZURL pObjectType and query for the corresponding interface through in Optional pointer to the URL that matches the byte QueryInterface. Stream. Syntax dwFlags HRESULT EndCreateCbject ( in Specifies resolution flags. See resolution flags section. in IMFAsyncResult pResult, 15 pProps out MF OBJECT TYPE* pCbjectType, in Holds the optional property store object that can contain out IUnknown ppunkObject other byte stream resolution related objects and flags. If a ); property store it not to be used, then this arg should be set to Parameters NULL. pResult pplUnknownCancelCookie in Pointer to an async result object passed in through out Holds the cancellation cookie that can be used in a call Invoke. o CancelObjectCreation(...) to abort the resolution process. pObjectType This parameter is optional and can be NULL. It is the out Specifies type of object that is being returned. Possible responsibility of the caller to Release the IUnknown pointer. values are: p Callback MF OBJECT MEDIASOURCE in Pointer to an async callback object MF OBJECT BYTESTREAM punkState MF OBJECT PLAYLIST 25 in Pointer to a context object associated with this call MF OBJECT TIMELINE Return Values MF OBJECT INVALID If the method succeeds, it returns S OK. If it fails, it returns an error ppObject code. out Holds the resolution object that is being returned. IMFByteStreamHandler:EndCreateObject Return Values The EndCreate0bject method completes the asynchronous call If the method succeeds, it returns S OK. If it fails, it returns an error 30 started in BeginCreateCobject and returns the resolution object code. obtained from the Supplied bytestream. IMFSchemeHandler:CancelObjectCreation Syntax The CancelObjectCreation method attempts to cancel the requested HRESULT EndCreateCbject( resolution associated with the provided cancel cookie. The cancel in IMFAsyncResult *pResult, cookie was obtained from the asynchronous BeginCreateCobject call. out MF OBJECT TYPE *pCbjectType, Note that canceling an existing resolution request is deemed to be 35 out IUnknown **ppObject “best-effort and might not truly yield the expected resource relinquishing effect as the resolution might have already completed Parameters and the user callback not invoked yet. pResult Syntax in Pointer to an async result object passed in through HRESULT CancelObjectCreation.( Invoke. in IUnknown *pIUnknownCancelCookie pObjectType ); 40 out Specifies type of object that is being returned. Possible Parameters values are: pIUnknownCancelCookie MF OBJECT MEDIASOURCE in Pointer to an IUnknown object that was provided by the MF OBJECT BYTESTREAM call BeginCreateCobject. MF OBJECT PLAYLIST Return Values MF OBJECT TIMELINE If the method succeeds, it returns S OK. If it fails, it returns an error 45 MF OBJECT INVALID code. ppObject out Holds the resolution object that is being returned. Return Values As previously noted, the byte stream handler 284 object If the method succeeds, it returns S OK. If it fails, it returns an error code. has the responsibility to identify the format of data presented 50 IMFByteStreamHandler:CancelObjectCreation to it in the form of a byte stream and return an appropriate The CancelObjectCreation method attempts to cancel the requested media Source. Upon completion of the process, a Byte Stream resolution associated with the provided cancel cookie. The cancel handler will either return a media source or fail with an error. cookie was obtained from the asynchronous BeginCreateCobject call. Note that canceling an existing resolution request is deemed to be In accordance with one implementation, these tasks are facili “best-effort and might not truly yield the expected resource tated using, in addition to the IUnknown interface, a IMF 55 relinquishing effect as the resolution might have already completed ByteStreamHandler interface 284. The IMFByteStreamHan and the user callback not invoked yet. Syntax dler interface 284 exposes five methods, which are defined as HRESULT CancelObjectCreation.( follows: in IUnknown *pIUnknownCancelCookie ); Parameters 60 pIUnknownCancelCookie IMFByteStreamHandler:BeginCreateCobject in Pointer to an IUnknown object that was provided by the The BeginCreateCobject method provides an asynchronous way for call BeginCreateCobject. creating a resolution object from the Supplied the byte stream. Return Values Syntax If the method succeeds, it returns S OK. If it fails, it returns an error HRESULT BeginCreateCbject( code. in IMFByteStream *pByteStream, 65 IMFByteStreamHandler:GetMaxNumberOfEytesRequiredForResolution in LPCWSTR pwszURL, The GetMextNumberOfEytesRequiredForResolution method should US 7,664,882 B2 23 24

-continued TABLE 1 return the maximum number of bytes the byte stream handler Resolution Object Requested requires to either create a resolution object or to signify that it cannot handle the byte stream. 5 Flag Description Syntax HRESULT GetMaxNumberOfEytesRequiredForResolution.( MF RESOLUTION The resolution process out QWORD *pdwBytes MEDIASOURCE should attempt to return ); a Media Source object Parameters MF RESOLUTION The resolution process pqwBytes 10 BYTESTREAM should attempt to return out The number of bytes required for resolution a Byte Stream object Return Values MF RESOLUTION The resolution process If the method succeeds, it returns S OK. If it fails, it returns an error PLAYLIST should attempt to return code. a “play list object MF RESOLUTION The resolution process 15 TIMELINE should attempt to return The following four tables briefly describe the resolution a timeline object flags that can be used in the BeginCreate(ObjectXXX calls described above:

TABLE 2

Resolution Behavior

Flag Description

MF RESOLUTION Enumerates though all byte stream handlers, if the CONTENT DOES NOT default byte stream handler for the extension fails. HAVE TO MATCH EXTENSION For example, suppose we have a WMV file with the extension "MP3. If you do not specify this flag, then the resolver will only try to invoke the MP3 handler on the file and it will fail since it is not an mp3 file. If this flag is specified, then after the failure, it will enumerate through all its known handlers to see if any of them will accept the file NOTE: This flag is mutually exclusive with the MF RESOLUTION KEEP BYTE STREAM ALIVE ON FAIL flag MF RESOLUTION KEEP BYTE STREAM This flag allows the bytestream that the resolver ALIVE ON FAIL creates during resolution to be cached (kept alive) on failure so it can be used on the Subsequent call. By specifying this flag, the caller agrees to attempt resolution again if the resolution fails. For example, Suppose we are trying to play back a network URL that points to a file with extension “.MP3 and resolution fails because the file is really a misnamed WMV file. If this flag is specified for the first resolution, then the network byte stream is cached (kept alive) by the resolver. On the Subsequent resolution, the user would specify the MF RESOLUTION CONTENT DOES NOT HAVE TO MATCH EXTENSION flag to allow handler enumeration. By using the MF RESOLUTION KEEP BYTE STREAM ALIVE ON FAIL, the networked URL is accessed once even though resolution is done twice. In this manner, the web server logs depict a more accurate acceSS count. NOTE: This flag is mutually exclusive with the MF RESOLUTION CONTENT DOES NOT HAVE TO MATCH EXTENSION flag US 7,664,882 B2 25 26

TABLE 3 TABLE 4

Functionality Behavior Access Mode RequestedC Flag Description Flag Description MF RESOLUTION READ The requested resource should be MF RESOLUTION Used to tell the created MF RESOLUTION WRITE R. should be METADATA MediaSource object that meta opened in write mode. data will probably be accessed. 10 MF RESOLUTION CREATE The requested resource should be MF RESOLUTION Used to tell the created opened in create mode. AUDIO VIDEO MediaSource object that audio and video data will probably be As previously noted, in accordance with one implementa accessed tion, the object selection module 214 uses function discovery MF RESOLUTION Used to tell the created 15 to find scheme and byte stream handlers. In one implemen NAVIGATION DATA MediaSource object that tation, function discovery only creates true COM objects. navigation data will Hence, in this implementation, all scheme and byte stream probably be accessed handlers will be truly CoCreate-able. The following sample code illustrates how to register scheme handlers using function discovery:

Static HRESULT RegisteryOYOSchemeHandlerWithFunctionDiscovery() { GUID YOYO SCHEME HANDLER COM CLASSID = {0}; HRESULT hr = S OK; PROPVARLANT propvarString: PropVariant|Init(&propvarString); do

// Create the Function Discovery Object ComSmartPtr-IFunctionDiscovery> spIFunctionDiscovery: hr = ::CoCreateInstance( uuidoffunctionDiscovery), NULL, CLSCTX ALL, uuidof IFunctionDiscovery ), (void**) &spIFunctionDiscovery); CHECKHR BRK(hr ); if Create the function instance builder

ComSmartPtr-IFunctionInstanceBuilder spIFunction InstanceBuilder; hr = spIFunctionDiscovery >TEMP RegisterFunctionInstance( FUNCTION DISCOVERY CATEGORY MF RESOLUTION SCHEME HANDLERS, LYOYO: Scheme Handler Function Instance, IVF DEFAULT, &spIFunctionInstanceBuilder); CHECKHR BRK (hr ); // Tell FD we only support IMFSchemeHandler interface and if the COM class ID to create hr = spIFunctionInstanceBuilder >SetInterface(&IID IMFSchemeHandler, NULL, &YOYO SCHEME HANDLER COM CLASSID, NULL ); CHECKHR BRK(hr ); // Get the Function Instance property Store ComSmartPtr- Internal Property Store > US 7,664,882 B2 27 28

-continued spIInternal Property Store; hr = spIFunctionInstanceBuilder >GetInternal PropertyStore( &spIInternal Property Store); CHECKHR BRK(hr ); // Store the Supported scheme propvarString.vt = VT LPWSTR. propvarString-pwszVal = Lyoyo:: hr = spIInternal Property Store->SetValue( MFPKEY Resolution FunctionDiscovery Scheme, &propvarString); CHECKHR BRK(hr ); Store the name propvarString.vt = VT LPWSTR. propvarString-pwszVal = LYOYO: Scheme Handler for MF: hr = spIInternal Property Store->SetValue( PKEY NAME, &propvarString); CHECKHR BRK (hr ); if Store a little comment propvarString.vt = VT LPWSTR. propvarString-pwszVal = L'This is the YOYO: Scheme handler for MF: hr = spIInternal Property Store->SetValue( PKEY Comment, &propvarString); CHECKHR BRK(hr ); if Commit the changes to Function Discovery hr = spIFunctionInstanceBuilder >TEMP CommitChanges(); CHECKHR BRK(hr ); while( FALSE): DestroyPropVariant(&propvarString); return (hr);

40 The following is sample code that demonstrates ho to reg ister a byte stream handler:

HRESULT RegisterXYZByteStreamHandlerWithFD() { HRESULT hr = S OK; PROPVARLANT propvarFDPropStoreEntry: PropVariantInit(&propvarFDPropStoreEntry); do

// Create the Function Discovery Object ComSmartPtr-IFunctionDiscovery> spIFunctionDiscovery: hr = ::CoCreateInstance( uuidof FunctionDiscovery ), NULL, CLSCTX ALL, uuidof IFunctionDiscovery ), (void**)&spIFunctionDiscovery); CHECKHR BRK(hr ); // Create the name of the bytestream hander and its function instance CMFStringW sByteStreamHandlerName(L*.xyz US 7,664,882 B2 29 30

-continued Byte Stream Handler); CMFStringW sByteStreamHandlerFunction InstanceName(L*.xyz Byte Stream Handler Function Instance'); if Create the function instance builder

ComSmartPtr-IFunctionInstanceBuilder spIFunction InstanceBuilder; hr = spIFunctionDiscovery >TEMP RegisterFunctionInstance( FUNCTION DISCOVERY CATEGORY MF RESOLUTION BYTESTR EAM HANDLERS, sByteStreamHandlerFunction InstanceName.PContents.( ), IVF DEFAULT, &spIFunctionInstanceBuilder): CHECKHR BRK(hr ); // Tell FD we only support IMFByteStreamHandler interface and if the COM class ID to create hr = spIFunctionInstanceBuilder >SetInterface(&IID IMFByteStreamHandler, NULL, &XYZ. BYTE STREAM HANDLER COM CLASSID, NULL ); CHECKHR BRK(hr ); // Get the Function Instance property Store ComSmartPtr- Internal Property Store > spIInternal Property Store; hr = spIFunctionInstanceBuilder >GetInternal PropertyStore( &spInternal PropertyStore); CHECKHR BRK(hr ); // Store the Supported file extension for this byte stream handler propvarFDPropStoreEntry.vt=VT LPWSTR. propvarFDPropStoreEntry.pwSzVal = L*.xyz': hr = spIInternal Property Store->SetValue( MFPKEY Resolution FunctionDiscovery File:Extension &propvarFDPropStoreEntry); CHECKHR BRK(hr );

Store the name propvarFDPropStoreEntry.vt=VT LPWSTR. propvarFDPropStoreEntry.pwszVal = (WCHAR *) shByteStreamHandlerName.PContents(); hr = spIInternal Property Store->SetValue( PKEY NAME, &propvarFDPropStoreEntry); CHECKHR BRK(hr ); // Store the max number of bytes required for resolution ComSmartPtr-IMFByteStreamHandlers spIMFByteStreamHandler; hr = CreateXYZByteStreamHandler? &spIMFByteStreamHandler); CHECKHR BRK(hr ); QWORD qwMaxNumberOfEytes = -1; hr = spIMFByteStreamHandler >GetMaxNumberOfEytesRequiredForResolution.( &qwMaxNumberOfEytes); CHECKHR BRK(hr ); propvarFDPropStoreEntry.vt=VT UI8: propvarFDPropStoreEntry.uhVal.QuadPart = qwMaxNumberOfEytes; hr = spIInternal Property Store->SetValue( US 7,664,882 B2 31 32

-continued MFPKEY Resolution FunctionDiscovery MaxNumberOfEy tesRequiredForResolution, &propvarFDPropStoreEntry); CHECKHR BRK(hr ); if Store a little comment propvarFDPropStoreEntry.vt=VT LPWSTR. propvarFDPropStoreEntry.pwSzVal = L'This is .xyz Byte Stream Handler that I wrote while I was watching Justice League.: hr = spIInternal Property Store->SetValue( PKEY Comment, &propvarFDPropStoreEntry); CHECKHR BRK(hr ); if Commit the new Function Instance to Function Discovery hr = spIFunctionInstanceBuilder >TEMP CommitChanges(); CHECKHR BRK(hr ); while( FALSE): DestroyPropVariant(&propvarFDPropStoreEntry ); return (hr);

FIG. 4 illustrates one operating environment 410 in which flash memory cards, digital video disks, Bernoulli cartridges, the various systems, methods, and data structures described random access memories (RAMS), read only memories herein may be implemented. The exemplary operating envi 30 (ROMs), and the like, may be used in the exemplary operating ronment 410 of FIG. 4 includes a general purpose computing environment. device in the form of a computer 420, including a processing A number of program modules may be stored on the hard unit 421, a system memory 422, and a system bus 423 that disk, magnetic disk 429, optical disk 431, ROM 424, or RAM operatively couples various system components include the 425, including an operating system 435, one or more appli system memory to the processing unit 421. There may be only 35 cation programs 436, other program modules 437, and pro one or there may be more than one processing unit 421. Such gram data 438. A user may enter commands and information that the processor of computer 420 comprises a single central into the personal computer 420 through input devices such as processing unit (CPU), or a plurality of processing units, a keyboard 40 and pointing device 442. Other input devices commonly referred to as a parallel processing environment. 40 (not shown) may include a microphone, joystick, game pad, The computer 420 may be a conventional computer, a distrib satellite dish, scanner, or the like. These and other input uted computer, or any other type of computer. devices are often connected to the processing unit 421 The system bus 423 may be any of several types of bus through a serial port interface 446 that is coupled to the structures including a memory bus or memory controller, a system bus, but may be connected by other interfaces, such as peripheral bus, and a local bus using any of a variety of bus 45 a parallel port, game port, or a universal serial bus (USB). A architectures. The system memory may also be referred to as monitor 447 or other type of display device is also connected simply the memory, and includes read only memory (ROM) to the system bus 423 via an interface, such as a video adapter 424 and random access memory (RAM) 425. A basic input/ 448. In addition to the monitor, computers typically include output system (BIOS) 426, containing the basic routines that other peripheral output devices (not shown), such as speakers help to transfer information between elements within the 50 and printers. computer 420, such as during start-up, is stored in ROM 424. The computer 420 may operate in a networked environ The computer 420 further includes a hard disk drive 427 for ment using logical connections to one or more remote com reading from and writing to a hard disk, not shown, a mag puters, such as remote computer 449. These logical connec netic disk drive 428 for reading from or writing to a remov tions may beachieved by a communication device coupled to able magnetic disk 429, and an optical disk drive 430 for 55 or a part of the computer 420, or in other manners. The remote reading from or writing to a removable optical disk 431 such computer 449 may be another computer, a server, a router, a as a CD ROM or other optical media. network PC, a client, a peer device or other common network The hard disk drive 427, magnetic disk drive 428, and node, and typically includes many or all of the elements optical disk drive 430 are connected to the system bus 423 by described above relative to the computer 420, although only a a hard disk drive interface 432, a magnetic disk drive interface 60 memory storage device 450 has been illustrated in FIG. 4. The 433, and an optical disk drive interface 434, respectively. The logical connections depicted in FIG. 4 include a local-area drives and their associated computer-readable media provide network (LAN) 451 and a wide-area network (WAN) 452. nonvolatile storage of computer-readable instructions, data Such networking environments are commonplace in office structures, program modules and other data for the computer networks, enterprise-wide computer networks, intranets and 420. It should be appreciated by those skilled in the art that 65 the Internal, which are all types of networks. any type of computer-readable media which can store data When used in a LAN-networking environment, the com that is accessible by a computer. Such as magnetic cassettes, puter 420 is connected to the local network 451 through a US 7,664,882 B2 33 34 network interface or adapter 453, which is one type of com 5. The method as recited in claim 1, wherein the second munications device. When used in a WAN-networking envi object is produced using a byte stream handler. ronment, the computer 420 typically includes a modem 454, 6. The method as recited in claim 1, wherein the first object a type of communications device, or any other type of com is produced by a scheme handler and the second object is munications device for establishing communications over the produced by a byte stream handler. wide area network 452. The modem 454, which may be 7. The method as recited in claim 1, wherein the first object internal or external, is connected to the system bus 423 via the is produced using a scheme handler selected from a list of two serial port interface 446. In a networked environment, pro or more scheme handlers. gram modules depicted relative to the personal computer 420, 8. The method as recited in claim 1, wherein the second or portions thereof, may be stored in the remote memory 10 object is produced using a byte stream handler selected from storage device. It is appreciated that the network connections a list of two or more byte stream handlers. shown are exemplary and other means of and communica 9. The method as recited in claim 1, wherein the first object tions devices for establishing a communications link between is produced using a scheme handler selected from a list of two the computers may be used. or more scheme handlers and the second object is produced Although some exemplary methods and systems have been 15 using a byte stream handler selected from a list of two or more illustrated in the accompanying drawings and described in the byte stream handlers. foregoing Detailed Description, it will be understood that the 10. The method as recited in claim 1, wherein the second methods and systems shown and described are not limited to object is produced using a byte stream handler selected from the particular implementation described herein, but rather are a list of byte stream handlers and wherein each byte stream capable of numerous rearrangements, modifications and Sub handler in the list has a selection value associated therewith. stitutions without departing from the spirit set forth and 11. The method as recited in claim 1, wherein the second defined by the following claims. object is produced using a byte stream handler selected from The invention claimed is: a list of byte stream handlers and wherein each byte stream 1. A method implemented in a component object-based handler in the list has a cost value associated therewith. multimedia framework of an operating system executing on a 25 12. The method as recited in claim 1, wherein the second computing device, the method comprising: object is produced using a byte stream handler selected from receiving a uniform resource locator (URL) as associated a list of byte stream handlers and wherein each byte stream with one of a plurality of applications requesting media handler in the list has a cost value associated therewith, the content; cost value indicating how many bytes must be read by the byte 30 stream handler in determining if the byte stream handler is identifying a scheme associated with the URL: appropriate for selecting the second object. selecting a first object operated to handle the identified 13. A computer-readable medium having computer-ex Scheme associated with the URL to access parameter ecutable instructions for performing operations in a compo data from a location specified by the URL: nent object-based multimedia framework encoded thereon, invoking the first object and determining whether the first 35 the computer-executable instructions, upon execution config object produces a second object according to a requested uring a processing unit to perform the operations comprising: media content type; receiving a uniform resource locator (URL) specifying a in an event the second object is produced according to the location of media content as associated with one of a requested media content type, then returning the second plurality of applications requesting media content; object to the one of the plurality of applications: 40 determining a scheme of the URL: in an event the second object comprises a media Source using the scheme to produce a byte stream object to handle object, then returning the media source object to the one the determined scheme associated with the URL to of the plurality of applications via a component object access parameter data; based pipeline, the pipeline comprising: using the byte stream object to generate a byte stream from the media source object performing format-specific 45 the media content; and parsing to produce one or more streams of data that using at least a portion of the byte stream to produce a are sent through the pipeline, and media Source object such that the media Source object: one or more media sinks rendering or archiving the performs format-specific parsing to produce one or more media content; and streams of data that are sent through the pipeline, and in an event the second object comprises a byte stream 50 one or more media sinks render or archive the media object and the requested media content type is a media content; Source object or a timeline object, then invoking one or based on the accessed parameter data, reads the media more selected byte stream handlers to generate the content of a given type from the location specified by the requested media Source object, wherein the byte stream URL, and generates one or more streams of media data handler is selected based in part on the byte stream 55 in a component object-based pipeline of the multimedia object, and returning the requested media source object framework, the pipeline further comprising one or more to the one of the plurality of applications via the com media sinks rendering or archiving the media content. ponent object-based pipeline. 14. The computer-readable medium as recited in claim 13, 2. The method as recited in claim 1, wherein the selection wherein the operation of using the byte stream additionally of the second object is additionally based on information 60 includes using a file extension indicated in the URL to select contained in the URL indicating a type of multimedia data of the source object. the media content. 15. The computer-readable medium as recited in claim 13, 3. The method as recited in claim 1, wherein the selection wherein the operation of producing a byte stream object of the second object is additionally based on Multipurpose includes choosing a scheme handler and using the chosen Internet Mail Extensions MIME data. 65 scheme handler to produce the byte stream object. 4. The method as recited in claim 1, wherein the first object 16. The computer-readable medium as recited in claim 13, is produced using a scheme handler. wherein the operation of producing a byte stream object US 7,664,882 B2 35 36 includes choosing a scheme handler from a list of scheme compile a second list of byte stream handlers, each of the handlers and using the chosen Scheme handler to produce the byte stream handlers in the second list not being associ byte stream object. ated with the type of the media content; 17. The computer-readable medium as recited in claim 13, invoke the byte stream handlers in the first list one at a time wherein the operation of producing a media source object until eitherabyte stream handler in the first list produces includes choosing a byte stream handler and using the chosen a media Source object or until all byte stream handlers in byte stream handler to produce the media Source object. the first list have been invoked without producing a 18. The computer-readable medium as recited in claim 13, media Source object; and wherein the operation of producing a media source object in an event all byte stream handlers in the first list have been includes choosing a byte stream handler from a list of byte 10 invoked and none of the invoked byte stream handler stream handlers and using the chosen byte stream handler to from the first list produced a media source object, then produce the media Source object. invoking each of the byte stream handlers in the second 19. The computer-readable medium as recited in claim 13, wherein the operation of producing a media source object list one at a time until either a either a byte stream includes choosing a byte stream handler from a list of byte handler from the second list produces a source object or stream handlers and using the chosen byte stream handler to 15 until all byte stream handlers in the second list have produce the media source object and wherein the list of byte invoked without producing a media Source object. stream handlers is ordered based on a selection values asso 27. The computer-readable medium as recited in claim 13, ciated with the byte stream handlers. wherein the operation of producing a media source object 20. The computer-readable medium as recited in claim 13, includes using a look-up process to: wherein the operation of producing a source object includes select a number of byte stream handlers; choosing a byte stream handler from a list of byte stream compile a first list of byte stream handlers, each of the byte handlers and using the chosen byte stream handler to produce stream handlers in the first list being associated with the the media source object and wherein the list of byte stream type of the media content, the byte stream handlers in the handlers is ordered based on a cost values associated with the first list being ordered according to cost values associ byte stream handlers. 25 ated with the byte stream handlers in the first list; 21. The computer-readable medium as recited in claim 13, compile a second list of byte stream handlers, each of the wherein the operation of producing a media source object byte stream handlers in the second list not being associ includes choosing a byte stream handler from a list of byte ated with the type of the media content; stream handlers and using the chosen byte stream handler to invoke the byte stream handlers in the first list one at a time produce the media Source object and wherein and each byte 30 in order until either a byte stream handler in the first list stream handler in the list has a cost value associated there produces a media Source object or until all byte stream with, the cost value indicating an amount of data that must be read by the byte stream handler in determining if the byte handlers in the first list have been invoked without pro stream handler is appropriate for producing the media Source ducing a media Source object; and object. in an event all byte stream handlers in the first list have been 22. The computer-readable medium as recited in claim 13, 35 invoked and none of the invoked byte stream handler wherein the operation of producing a media source object from the first list produced a source object, then invoking includes using a look-up process to select a Subset of byte each of the byte stream handlers in the second list one at stream handlers from a set of byte stream handlers and using a time until eithera eithera byte stream handler from the one of the subset of byte stream handlers to produce the media second list produces a media Source object or until all Source object. 40 byte stream handlers in the second list have invoked 23. The computer-readable medium as recited in claim 13, without producing a media source object. wherein the operation of producing a source object includes 28. The computer-readable medium as recited in claim 13, using a look-up process to: wherein the operation of producing a byte stream object select a number of byte stream handlers; and includes using a look-up process to: invoke the selected byte stream handlers one at a time until 45 select a number of scheme handlers; and a byte stream handler produces a media Source object. invoke the scheme handlers in the list one at a time until a 24. The computer-readable medium as recited in claim 13, Scheme handler produces a byte stream object. wherein the operation of producing a media source object 29. The computer-readable medium as recited in claim 13, includes using a look-up process to: wherein the operation of producing a byte stream object select a number of byte stream handlers; and 50 includes using a look-up process to: invoke the selected byte stream handlers one at a time in a select a number of scheme handlers; and predetermined order until a byte stream handler pro invoke the scheme handlers in the list one at a time in a duces a media source object. predetermined order until a scheme handler produces a 25. The computer-readable medium as recited in claim 13, wherein the operation of producing a media source object byte stream object. includes using a look-up process to: 55 30. The computer-readable medium as recited in claim 13, select a number of byte stream handlers; and wherein the operation of producing a media source object invoke the byte stream handlers one at a time in a prede includes using a look-up process to: termined order based on cost values associated with the select a number of scheme handlers; selected byte stream handlers untila byte stream handler compile a list of scheme handlers, each of the scheme produces a media source object. 60 handlers in the list of scheme handlers being associated 26. The computer-readable medium as recited in claim 13, with the scheme of the URL: wherein the operation of producing a media source object invoke the scheme handlers in the list of scheme handlers includes using a look-up process to: one at a time until either a byte stream object is pro select a number of byte stream handlers; duced, a media Source object is produced, or all scheme compile a first list of byte stream handlers, each of the byte 65 handlers in the list of scheme handlers have been stream handlers in the first list being associated with the invoked and neither a byte stream object nor a media type of the media content; Source object have been produced; in an event either a US 7,664,882 B2 37 38 media Source object or a byte stream object has been 39. A system of a component object-based multimedia produced, then determining if an application has framework comprising: requested a media source object; means for receiving a uniform resource locator (URL) as in an event an application has requested a media Source associated with one of a plurality of applications object, then providing the media source object to the 5 requesting media content; application; and means for identifying a scheme associated with the URL: in an event if the application has not requested a media means for selecting a scheme handler based on the identi Source object, then compile a list of byte stream handlers fied scheme, the scheme handler producing a byte and invoking the byte stream handlers in the first list one stream object operable to handle the identified scheme at a time until either a byte stream handler in the list 10 associated with the URL to access parameter data from produces a media source object or until all byte stream the location pointed to by the URL and produce a byte handlers in the first list have been invoked without pro stream from the accessed parameter data; and ducing a media Source object. means for selecting a byte stream handler based on the 31. A computerized system including: accessed parameter data, the byte stream handler pro an object selection module in a component object-based 15 ducing a media source object operable to read the media multimedia framework operable to: content of a given type, from the location specified by receive a uniform resource locator (URL) as associated the URL, and operated to generate data in a component with one of a plurality of applications requesting media object-based pipeline, the pipeline further comprising content; one or more media sinks rendering or archiving the determine a scheme of the URL specifying a location of media content. media content; 40. The system as defined in claim 39, further comprising use the scheme to produce a byte stream object to handle a lookup means for producing a list of Scheme handlers, the determined scheme associated with the URL to wherein the means for selecting the scheme handler selects access parameter data; the scheme handler from the list of scheme handlers. use the byte stream object to generate a byte stream from 25 the media content; and 41. The system as defined in claim 39, further comprising use a portion of the byte stream to produce a media Source a lookup means for producing a list of byte stream handlers, object that, based on the accessed parameter data, reads wherein the means for selecting the byte stream handler the media content of a given type from the location selects the byte stream handler from the list of byte stream specified by the URL, and generates one or more streams handlers. of media data in a component object-based pipeline of 42. The system as defined in claim 39, wherein the means the multimedia framework, the pipeline further compris for selecting a scheme handler selects the scheme handler in ing one or more media sinks rendering or archiving the response to a request from an application. media content. 43. The system as defined in claim 39, wherein the means 32. The computerized system as recited in claim 31, for selecting a byte stream handler selects the byte stream wherein the byte stream object is produced using a scheme 35 handler in response to a request from an application. handler. 44. The system as defined in claim 39, wherein the media 33. The computerized system as recited in claim 31, source object produced by the byte stream handler is wherein the media source object is produced using a byte employed as a component in a multi-component media pro stream handler. cessing pipeline. 34. The computerized system as recited in claim 31, 40 45. The system as defined in claim 39, wherein the media wherein the byte stream object is produced using a scheme source object produced by the byte stream handler is handler that is selected from a list of scheme handlers, the list employed as a component in a multi-component media pro being selected based on the scheme of the URL. cessing pipeline in a media engine. 35. The computerized system as recited in claim 31, wherein the media source object is produced using a byte 45 46. The system as defined in claim 39, wherein the means stream handler that is selected from a list of byte stream for selecting a scheme handler selects the scheme handler in handlers, the list being selected based on a byte stream gen response to a request from an application and wherein the erated from data at the location indicated by the URL and a media source object produced by the byte stream handler is portion of the URL. employed as a component in a multi-component media pro 36. The computerized system as recited in claim 31, 50 cessing pipeline in a media processing module that is an wherein the byte stream object is produced using a scheme operational module in a operating system. handler that is selected from a list of scheme handlers, the list 47. The system as defined in claim 39, wherein the means being selected based on the scheme of the URL and ordered for selectingabyte stream handler employs a lookup module. based on cost values associated with each of the scheme 48. One or more computer-readable storage media having handlers in the list. 55 computer-executable instructions encoded thereon, the com 37. The computerized system as recited in claim 31, puter-executable instructions, upon execution configuring a wherein the operation of producing a media source object processing unit to perform operations comprising: includes using a look-up process to: responsive to receiving a uniform resource locator (URL) select a number of byte stream handlers; and for media content from one of a plurality of applications invoke the selected byte stream handlers one at a time until 60 executing on an operating system of a computing device, a byte stream handler produces a media Source object. Selecting a media source object using an object-based 38. The computerized system as recited in claim 31, multimedia framework of the operating system, wherein wherein the operation of producing a byte stream object the selecting comprises identifying a scheme associated includes using a look-up process to: with the URL: Select a number of scheme handlers; and 65 identifying one or more available scheme handler objects invoke the scheme handlers in the list one at a time until a operable to read information associated with the scheme Scheme handler produces a byte stream object. of the URL: US 7,664,882 B2 39 40 invoking the scheme handler object operable to handle the based media pipeline that comprises the media source identified scheme associated with the URL to access object that performs format specific parsing to pro parameter data from a location specified by the URL, duce one or more streams of media data and providing based on the accessed parameter data, generating a selected the one or more streams of media data to a media sink media Source object; and 5 object, the media sink object rendering or archiving providing the selected media source object to the one of the media content. the plurality of applications executing on the operat ing system of the computing device via an object