USOO7743019B2

(12) United States Patent (10) Patent No.: US 7,743,019 B2 Shah et al. (45) Date of Patent: Jun. 22, 2010

(54) SYSTEMS AND METHODS FOR PROVIDING 5,774,717 A 6/1998 Porcaro et al. SYNCHRONIZATION SERVICES FOR UNITS 5,806,074 A 9, 1998 Souder et al...... 707/21 OF INFORMATION MANAGEABLE BY A 5,832,496 A * 1 1/1998 Anand et al...... 707/102 HARDWAREASOFTWARE INTERFACE 5,842,213 A 1 1/1998 Odom et al. ... 707/100 SYSTEM 5,893,106 A 4, 1999 Brobst et al...... 707/102 (75) Inventors: Ashish Shah, Sammamish, WA (US); Darshatkumar Shah, Bellevue, WA (Continued) (US); Irena Hudis, Bellevue, WA (US); Lev Novik, Bellevue, WA (US); Vivek FOREIGN PATENT DOCUMENTS Jawahir Jhaveri, Seattle, WA (US) EP O974.895 1, 2000 (73) Assignee: Corporation, Redmond, WA (US) (Continued) (*) Notice: Subject to any disclaimer, the term of this OTHER PUBLICATIONS past lSr.listed under 35 (Oliver Ibelshauser, “The WinFS For Windows Long M YW- (b) by ayS. horn: Faster & Smarter” Jun. 17, 2003, pp. 1-7.* (21) Appl. No.: 10/692,515 (Continued) (22) Filed: Oct. 24, 2003 Primary Examiner Hosain TAlam Assistant Examiner Usmaan Saeed (65) Prior Publication Data (74) Attorney, Agent, or Firm Woodcock Washburn LLP US 2005/OO44108A1 Feb. 24, 2005 (57) ABSTRACT Related U.S. Application Data (63) Continuation-in-part of application No. 10/646,575, Several embodiments of the present invention employ syn filed on Aug. 21, 2003 chronization adapters for synchronizing information between 9. 41. “WinFS and non-"WinFS data sources. Examples of adapt (51) Int. Cl. ers include an adapter that synchronizes address book infor G06F 7/00 (2006.01) mation between a “WinFS contacts folder and a non-WinFS G06F 7700 (2006.015 mailbox. In these instances, adapter developers might use the (52) U.S. Cl...... 707,610.707,640.707661.707/674 CESISWinFS synchronization ity,p core servicesE. the APIWins described R herein (58) Field of Classification Search ...... 707/205, 70704. 203 Zatoncode between platform the in “WinFSEder, developSchema schenaand the transistan non-"WinFS S lication file f let h hist s data source Schema. Additionally, the adapter developer pro ee appl1cauon Ille Ior complete searcn n1Story. vides protocol Support for communicating changes with the (56) References Cited non-"WinFS data source. A synchronization adapter is invoked and controlled by using the synchronization control U.S. PATENT DOCUMENTS ler API and reports progress and errors using this API. 5,388,257 A 2, 1995 Bauer ...... 707/1 5,742,813 A * 4, 1998 Kavanagh et al...... 707/8 28 Claims, 34 Drawing Sheets

System (Non-WinFS) 3606

Application using WinFS Sync Ns - 3666 WinFS Sync ConfiglControl - 3884 Sync Adapter s 3644 Ox 3662 3652

Sync AP sync API

WinFS WinFS 3622 Data Store ata Stre 3612 3614 3842 WinFS to WinFS sync WinFS to WinFS sync Core Sync Services Core Sync Services (WinFS to non-WinFS) (WinFS to non-WinFs) 3824 7 System 364 3602 WinFS System III (WinFS US 7,743,019 B2 Page 2

U.S. PATENT DOCUMENTS 7,191,299 B1* 3/2007 Kekre et al...... T11 162 7,206,788 B2 4/2007 Horvitz et al. 5,900,870 A 5/1999 Malone et al...... 345,333 7,219,327 B1 5/2007 Jacobs et al. 5.937,189 A 8, 1999 Branson et al. 7,240,072 B2 * 7/2007 McKnight et al...... 707/103 Y 5.937,402 A 8/1999 Pandit ...... 707/4 7,249,117 B2 * 7/2007 Estes ...... 706.52 5.937,406 A * 8/1999 Balabine et al...... 7O7/1OO 7,272.598 B2 * 9/2007 Cunningham et al...... 707/3 6,006,234. A 12/1999 Govindarajan et al. .. 707/103 R 2002fOO6737O A1 6/2002 Forney et al. 6,047.291 A 4/2000 Anderson et al...... 707/103 2002/0069.192 A1* 6/2002 Aegerter ...... 707/1 6,085,192 A 7/2000 Mendez et al...... TO7/10 2002/009 1702 A1* 7/2002 Mullins ...... 7O7/1OO 6,108,004 A 8, 2000 Med1 ...... 345,346 2002/0099.713 A1* 7/2002 Fernandez et al...... 7O7/1OO 6,112,024 A 8, 2000 Almond et al. 395,703 2002/0120763 A1* 8, 2002 Miloushev et al. . TO9/230 6,151,606 A 1 1/2000 Mendez ...... 707/2O1 2002/0152422 Al 10/2002 Sharma et al...... T13/13 6,189,000 B1* 2/2001 Gwertzman et al...... 707/1 2002/0156792 A1 10, 2002 Gombocz et al...... 707/103 6,199,195 B1 3/2001 Goodwin et al...... 717, 1 2002/0174180 A1* 11/2002 Brown et al...... TO9,203 6,240,414 B1 5/2001 Beizer et al...... 707/8 2002/0174417 A1 1 1/2002 Sijacic et al. 6,263.339 B1 7/2001 Hirsch 2002/0177993 A1* 11/2002 Veditz et al...... TO4/8 6,317,754 B1 * 1 1/2001 Peng ...... 707/2O3 2002/0184163 Al 12/2002 Lotter et al...... 7O7/4 6,324,533 B1 1 1/2001 Agrawal et al...... 707.3 2002fO184401 All 12/2002 Kadel et al. 6,338,056 B1 1/2002 Dessloch et al...... 707/2 2002/0194388 A1* 12/2002 Boloker et al...... TO9,310 6,343,287 B1 1/2002 Kumar et al...... 707/4 2002fO198891 A1 12/2002 Li et al...... 707/102 6,370,541 B1 4/2002 Chou et al. .... 707/103 2003/0083948 A1* 5/2003 Rodriguez et al. . ... 705/26 6,418.438 B1* 7/2002 Campbell ...... TO7/8 2003/0088654 A1* 5/2003 Good et al...... 709,223 6,430,564 B1 8/2002 Judge et al...... 707/100 2003/0101190 A1 5/2003 Horvitz et al. 6,438,545 B1 8/2002 Beauregard et al...... 707/6 2003/0110188 A1* 6/2003 Howard et al...... 707/2OO 6,442,548 B1* 8/2002 Balabine et al...... TO7/10 2003. O144849 A1 7/2003 Kakivaya et al...... TO5/1 6,446,092 B1 9/2002 Sutter ...... 707/2O3 2003/0172368 A1* 9/2003 Alumbaugh et al...... 717/106 6,473,851 B1 10/2002 Plutowski 2004/0003091 A1 1/2004 Coulthard et al. 6,477.527 B2 11/2002 Carey et al...... 707/4 2004/0024795 A1 2/2004 Hind et al...... 707,204 6,477.564 B1 1 1/2002 Freyssinet et al. . 709f2O2 2004/0025 11.0 A1 2/2004 Hu ...... 71.5/5OO 6,519,597 B1 2/2003 Cheng et al...... 707/10 2004/0031058 A1* 2, 2004 Reisman 72.5/112 6,553,391 B1 4/2003 Goldring 2004.0068523 A1* 4, 2004 Keith et all 707/2OO 6,556,983 B1 4/2003 Altschuler et al...... 7O6/55 2004/OO73560 A1 4/2004 Edwards 707/2OO 6,578,046 B2 6/2003 Chang et al...... 707/103 2004f0078568 A1* 4, 2004 Pham et all ... 713,165 6,601.234 B1* 7/2003 Bowman-Amuah ...... 717/108 2004/0177319 A1* 9, 2004 Horn ...... 715/5011 6,643,652 B2 11/2003 Helgeson et al...... TO7/10 2004/O193707 A1* 9, 2004 Alam et al...... 709,223 6,671,757 B1 12/2003 Multer et al...... T10/100 2004/0193952 A1* 9/2004 Narayanan et al. . ... 714f13 6,694,321 B1 2/2004 Berno 2004/0199521 A1 10/2004 Anglin et al...... 707/103 6,694,336 B1* 2/2004 Multer et al...... 707/2O1 2004/0215858 A1* 10/2004 Armstrong etal T10/200 6,701,314 B1 3/2004 Conover et al...... 707/7 2004/0268240 Al 12/2004 Vincent ...... 715,513 6,704,743 B1 3/2004 Martin ...... TO7/103 R 2005, OO15361 A1 1/2005 Payton et al. 6,708,221 B1 3, 2004 Mendez et al. . ... 709/248 2005, OO15363 A1 1/2005 Dessloch et al. 6,714,943 B1* 3/2004 Ganesh et al...... TO7 104.1 2005/0015663 A1* 1/2005 Armangau et al...... T14? 15 6,728,719 B1 4/2004 Ganesh et al...... 707/100 2005, OO44089 A1 2, 2005 Wu et al. 6,738,789 B2 5/2004 Multer et al. ... TO7,201 2005/0065977 A1 3/2005 Benson et al...... TO7 104.1 6,757,696 B2 6/2004 Multer et al...... 707/2O1 2005, 012543.0 A1* 6, 2005 Souder et al. ... 7O7/1OO 6,763,350 B2 7/2004 Agrawal et al...... 707/100 2005/0273759 A1* 12/2005 Lucassen et al. 717/105 6,763,361 B1 7/2004 Poskanzer 2006/0173873 A1* 8/2006 Promptet al. ... 7O7/1OO 6,772,178 B2 8/2004 Mandal et al...... 707,204 2007/0050620 A1 3/2007 Pham et al...... T13,165 6,801,604 B2 10/2004 Maes et al...... 379/88.17 6,810,516 B2 10/2004 Lauris FOREIGN PATENT DOCUMENTS 6,823,349 B1 * 1 1/2004 Taylor et al...... 707,204 6,839,721 B2 1/2005 Schwols WO WO O2/O75539 A2 9, 2002 6,842,906 B1* 1/2005 Bowman-Amuah ...... T19,330 WO WO 2005/024665 3, 2005 6,851,089 B1 2/2005 Erickson et al...... 715,513 WO WO 2005/029363 3, 2005 6,857.053 B2 2/2005 Bolik et al...... T11 162 6,889,231 B1* 5/2005 Souder et al...... 707/104.1 OTHER PUBLICATIONS 6,895,586 B1 5/2005 Brasher et al...... T19, 313 6,952,704 B2 10/2005 MacLeod et al. Andrews, T. et al., "Combining Language and Database Advances in 6.961,723 B2 11/2005 Faybishenko et al...... 707/100 an Object-Oriented Development Environment”, OOPSLA Proceed 6,968,344 B2 11/2005 Bahulkar et al. ings, Oct. 4-8, 1987, 430-440. 6,976,027 B2 * 12/2005 Cutlip ...... 707/101 Beard, et al., “Multilevel and Graphical Views of '. 6,990,513 B2 1/2006 Balfiore et al...... TO9,203 Research and Technology Advances in Digital Libraries, 1998, 256 6,999,956 B2 2/2006 Mullins 265. 7,031,973 B2 4/2006 Natarajan et al...... 707/102 Beitner, N.D. et al., “Multimedia Support and Authoring in Micro 7,031,974 B1 * 4/2006 Subramaniam ... 707/102 cosm: An Extended Model”. Department of Electronics and Com 7,043,481 B2 5/2006 Mullins et al...... 707/10 puter Science, University of Southampton, 12 pages. 7,099,932 B1 8/2006 Frenkel et al...... 709,223 Berg, C. How Do I Create Persistent Java Objects? Dr. Dobb's 7,158,962 B2 1/2007 Nelson ...... 707/2 Journal, 1997. 22(4), 98-101. 7,162.469 B2 1/2007 Anonsen et al. Bhattacharya, S. et al., "Coordinating Backup/Recovery and Data 7, 177,843 B2 2/2007 Nguyen et al...... 705/51 Consistency Between Database and File Systems'. International 7, 177,865 B2 2/2007 Sasaki ...... 707/9 Conference On Management of Data and Symposium on Principles of 7,178,100 B2 2/2007 Call ...... TO4/9 Database Systems, Proceedings of the 2002 ACM SIGMOD Interna 7,181,450 B2 * 2/2007 Malloy et al...... 707/4 tional Conference on Management of Data, 2002, 500-511. US 7,743,019 B2 Page 3

Biliris, A., “The Performance of Three Database Storage Structures Kaneko, K, et al., “Design of 3D CG DataModel of Move Animation for Managing Large Objects”, ACM SIGMOD, 1992, 276-285. Database System”. Advanced Database Research and Development Booch, G. Benjamin/Cummings Publishing Co, "Object-Oriented Series, vol. 3, Proceedings of the Second Far-East Workshop On Analysis and Design with Applications”, 1994, 155, 156, 179-183. Future Database Systems, 1992, 364-372. Bracchi et al., “Binary Logical Associations in Data Modelling'. Kaneko, K. et al., Towards Dynamics Animation on Object-Oriented Modelling in Data Base Management Systems G.M. Nijssen, (ed); Animation Database System Move, Advanced Database Research North Holland Publishing Company: 1976, 125-147. and Development Series, vol. 4. Database Systems for Advanced Buneman, P. et al., Inheritance and Persistence in Database Program Applications 1993, 3-10. ming Languages, ACM, 1986, 4-15. Katz, R.H., “Toward a Unified Framework for Version Modeling in Chien, A.A., “Concurrent Aggregates (CA)—Design and Experience Engineering Databases”. ACM Computing Surveys, 1990. 22(4), with a Concurrent Object-Oriented Language Based on Aggre 375-408. gates'. J. Parallel and Distributed Computing, 1995. 25(2), 174-196. Kawabe, S. et al., “A Framework for 3D Modeling Constraint-Based Chryssostomidis, Chryssosiomos, et al. 'Geometric Modeling Issues Description and Non-Manifold Geometric Modeling”. A Collection in Computer Aided Design of Marine Structures”, MTS Journal, of Contributions based on Lectures Presented at the 2d Toyota Con 22(2) pp. 15-33. ference, Organization of Engineering Knowledge for Product Model “Computervision Launches Design Automation Development Plat ling in Computer Integrated Manufacturing, Japan, Oct. 2-5, 1988, form for Windows'. PR Newswire, Financial News, Jan. 10, 1995. 325-357. D'Andrea, A. et al., “Unisql's Next Generation Object-Relational Kempfer, L., “CADJumps on Windows 3.1 Bandwagon'. Computer Database Management System”, ACM SIGMOD Record, Sep. 1996, Aided Engineering, 1993, 24-25. 25(2), 70-76. Khan, L. et al. A Performance Evaluation of Storing XML Data in Darby, C., Object Serialization in Java 1.1. Making Objects Persis Relational Database Management Systems, WIDM, 2001, 31-38. tent, WEB Techniques, 1997, 2(9), 55, 58-59. Khoshafian, S. et al., “Object Identify”, OOPSLA '86, 1986, 21, 406 “Developer's Guide to Apple Data Detectors-For Version 1.0.2, C) 416. Apple Computer, Inc., 1997, 1-34. Kiesling, R., "ODBC in UNIX Environments'. Dr. Dobb's Journal, Dietrich, Walter C., Jr., et al., “TGMS: An Object-Oriented System Dec. 2002, 27(12), 16-22. for Programming Geometry”. Software-Practice and Experience, King et al., “TriStarp- An Investigation into the Implementation and Oct. 1989, 19(10),979-1013. Exploitation of Binary Relational Storage Structures'. Proc. 8.sup.th Dobashi, Y. et al. “Skylight for Interior Lighting Design'. Computer BNCOD(British National Conference on Data Bases), pp. 64-84 Graphics Forum, 1994, 13(3), C85-C96. (York 1990). Dorsch, Jeff, "Accel Signs with IBM Altium PCB Unit-Accel Tech Krouse, J.K. “Geometric Models for CAD/CAM', Machine Design, nologies Acquires the Assets to Altium's P-CAD Business Unit', Jul. 24, 1990, 99-105. EDA Licensing, Electronic New, Jan. 16, 1995, 4 pages. LeBlanc, Andrew R. et al., “Design Data Storage and Extraction Fegaras, Leonidas, "Optimizing Object Queries Using an Effective Using Objects”. Concurrent Engineering. Research and Applica Calculus'. ACM Transactions On Database Systems, Dec. 2000, tions, 1993, 1, 31-38. 25(4), 457-516. Leontiev, Y. et al. “On Type Systems for Object-Oriented Database Findler, R.B. et al., Contract Soundness for Object-Oriented Lan Programming Languages'. ACM Computing Surveys, Dec. 2002, guages ACM Conference on Object-Oriented Programming Sys 34(4), 409-449. tems, Languages, and Applications, OOPSLA, 2001, 15 pages. Lim, J.B. et al. “Transaction Processing in Mobile, Heterogeneous Foley et al., Computer Graphics: Principles and Practices, Second Database Systems', IEEE Trans. On Knowledge and Data Engineer Edition, Addison-Wesley Publishing Company, 1990, Ch. 5 and 9. ing, 2002, 14(6), 1330-1346. pp. 201-283. Friis, A.-Christensen, et al. "Geographic Data Modeling: Require Mallet, S. et al., “Myrtle: A Set-Oriented Meta-Interpreter Driven by ments and Research Issues in Geographic Data Modeling.” Nov. a Relational Trace for Deductive Databases Debugging. Lecture 2001, Proceedings of the 9th ACM International Symposium on Notes in Computer Science, 1999, 1559, 328-330. Advances in Geographic Information Systems, 2-8. Mariani, J. A., Oggetto: “An Object Oriented Database Layered on a Fuh, Y-C. et al., “Implementation of SQL3 Structured Types with Triple Store'. The Computer Journal, 1992, 35(2), 108-118. Inheritance and Value Substitutability”. Digital Symposium Collec McMahon, L.E. "SED-A Non-Interactive TextEditor'. Bell Labora tion, 2000, Abstract only, 2 pages, www.acm.org/sigmod/disc/ tories, Aug. 15, 1978, 10 pages. p implementationoyostw.htm. “Mechanical Design Software (Buyers Guide)”. Computer-Aided Garret, J.H., Jr. et al., “An Object Oriented Environment for Repre Engineering, Dec. 1993, 12(12), 32-36. senting Building Design and Construction Data'. Advanced Con Melton, J. etal, “SQL and Management of External Data'. SIGMOD struction Technology Center, Jun. 1989, Document No. 89-37-04. Record, Mar. 2001, 30(1), 70-77. 1-34. Mitchell, W.J., “The Logic of Architecture', Massachusetts Institute Godoy Simóes, M. et al., “A RISC-Microcontroller Based of Technology, 1990, 139-143. Photovoltaic System for Illumination Applications'. APEC 2000. Navathe, S.B., “Evolution of Data Modeling for Databases.” Com Fifieenth Annual IEEE Applied Power Electronics Conference and munications of the ACM, Sep. 1992, 35(9), 112-123. Exposition, Feb. 6-10, 2000, 2, 1151-1156. Nelson, M. et al., “Generic Support for Caching and Disconnected Goscinski, A. “Distributed Operating Systems The Logical Design”. Operation', 4th Workshop On Workstation Operating Systems, Oct. Addison-Wesley, 1991, 306–313. 1993, 61-65. Harrison, C.J. et al., “Structure Editors: User-Defined Type Values Nijssen, G.M. et al., “Conceptual Schema and Relational Database and Type Inference", IEEE, 2000, 241-247. Design, A Fact Oriented Approach'. Department of Computer Sci Haverlock, K., "Object Serialization, Java, and C++”. Dr. Dobb's ence, University of Queensland, Prentice Hall, 10-33, 42-43, 48-51, Journal, 1998, 23(8), 32, 34, 36-37. 156-17O. Hay, David C. " Patterns: Convention of Thought”. Oracle 9i SQL Reference, Release 2 (9.2), Mar. 2002, 13-89 to 13-90. (Dorset House Publishing, New York, NY 1996, 47-67. 235-259. Orenstein, J, et al. "Query Processing in the Object Store Database Hernandez, M.A. et al. “The Merge/Purge Problem for Large Data System”, ACM SIGMOD International Conference on Management bases, International Conference on Management of Data and Sym of Data, Jun. 1992, 21(2),403-412. posium on Principles of Database Systems'. Proceedings of the 1995 Ottogalli. F.G. et al., “Visualisation of Distributed Applications for ACM SIGMOD International Conference On Management of Data, Performance Debugging. Lecture Notes in Computer Science, Jan. 1995, 127-138. 2001, 2074, 831-840. Hsiao, H.I. et al., “DLFM: A Transactional Resource Manager', Pachet, et al., “A Combinatorial Approach to Content-Based Music SIGMOD, Proceedings of the 2000 ACM SIGMOD International Selection'. Multimedia Computing and Systems, Jun. 7, 1999, 457 Conference On Management of Data, 2000, 518-528. 462. US 7,743,019 B2 Page 4

Papiani, M. et al., “A Distributed Scientific Data Archive Using the Mazzola Paluska, J. et al., “Footloose: A Case for Physical Eventual Web, XML and SQL/MED', SIGMOD Record, Sep. 1999, 28(3), Consistency and Selective Conflict Resolution”. Proceedings of the 56-62. 5' IEEE Workshop on Mobile Computing Systems and Applications, Powell, M., "Object, References, Identifiers, and Equality White 2003, 170-179. Paper”, (Jul. 2, 1993), OMG TC Document 93.7.5, 1-24. Huang, Yun-Wu. et al., “Lightweight Version Vectors for Pervasive Prosise, J., “2-D Drafting: Why Pay More?”, PC Magazine. The Computing Devices', IEEE, 2000, 43-48. Independent Guide to IBM-Standard Personal Computing, 1993, Ramsey, N. et al., “An Algebraic Approach to File Synchronization'. 12(4), 255-289. Sofiware Engineering Notes, Association for Computing Machinery, Reiner, A. et al., “Benefits of X-based Three-Tier Client/ Sep. 2001, 26(5), 175-185, XP002295139. Model with ESRI Applications'. Virtual Solutions, 1995, 9 pages. “SyncML Sync Protocol, Version 1.0”. Syncinl Consortium, 2000, Read, III, B.C., “Developing the Next Generation Cockpit Display XP-002217356, 60 pages. System”, IEEE Aerospace and Electronics Systems Magazine, 1996, Berenson, H. "A Critique of ANSI SQL Isolation Levels”, SIGMOD 11(10), 25-28. RECORD, 1995, 24(2), 10 pages. Rouse, N.E., "CAD Pioneers are Still Trailblazing”. Machine Gray, J. et al., “The Dangers of replication and a Solution'. SIGMOD. Design, Oct. 22, 1987, 59(25), 117-122. 1996, 25(2), 173-182, XP-002146555. Roussopoulos, N. et al., “Using Semantic Networks for Data Base Greenwald, R. et al., “Oracle Essentials: Oracle 8 & Oracle 8i’, Multi Management”. Proceedings of the I Supplemental VLDB Confer User Concurrency, 1999, Ch. 7, 7 pages, XP-002312028. ence, 1975, 144-172. Guy, R.G. et al., “Implementation of the Ficus Replicated File Sys Santos, J.L.T. et al., “Computer Aided Mechanical Engineering tem”, proceedings of the Summer USENIX Conference, Jun. 1990, Design Environment for Concurrent Design '. Proceedings 63-71 of the 1993 ITEC Workshop on Concurrent Engineering, May 4-6, Helal, S. et al., "A Three-tier Architecture for Ubiquitous Data 1993, Simulation in Concurrent Engineering, 71-83. Access”. Computer Systems and Applications ACS/IEEE, Jun. 2001, Seshadri, P., “Enhanced Abstract Data Types in Object-Relational 177-180, XPO 1055.1207. Databases”. The VLDB Journal, The International Journal on Very Kistler, J.J. et al., “Disconnected Operation in the File System'. Large Databases, 1998, 7, 130-140. ACM Transactions on Computer Systems, Feb. 1992, 10(1), 3-25, Simon, A.R., Strategic Database Technology. Management for the XPOOO323223. Year 2000, 1995, pp. 6-9, 14-17, 55-57, Morgan Kaufmann Publish Kistler, J.J. et al., “Increasing File System Availability through Sec S. ond-Class Replication”, IEEE, 1990, 65-69, XPO10021244. Sreenath, N., “A Hybrid Computation Environment for Multibody Plattner, C. et al., “Ganymed: Scalable Replication for Transactional Simulation'. Mathematics and Computers in Simulation, 1992, 121 Web Applications', IFIP International federation for Information 140. Processing, 2004, 155-174, XP-002370 197. Singhal, A. et al., “DDB. An Object Design Data Manager for VLSI Seshadri, P. et al., “SQLServer for Windows CE-A Database Engine CAD'. Association for Computer Machinery, 1993, 467-470. for Mobile and Embedded Platforms”, IEEE Comput. Soc., 2000, Stevens, T., “Value in 3-D, Industry Week, Jan. 8, 1995, 45-46. 642-644, XP 010378761. Stonebraker, M., “The Case for Partial Indexes'. SIGMOD Record, Shapiro, M. et al., “Managing Databases with Binary Large Objects', 1989, 18(4), 4-9. IEEE, 1999, 185-193, XP-000922048. Strickland, T.M.. “Intersection of Relational and Object”. Proceed Barker, J., “Beginning Java Objects.” Wrox Press Ltd., 2000, pp. 1 ings of the AM/FM International Conference XVII, Mar. 14-17, 1994, and 94. 69-75. Bernstein et al., “The Microsoft Repository.” Proceedings of the 23 Sutherland, J. et al., “The Hybrid Object-Relational Architecture VLDB Conference, 1997, http://citeSeerist.psu.edu/ (HORA), An Integration of Object-Oriented and Relational Technol bernstein 97 microsoft.html. ogy”, Applied Computing. States of the Art and Practice, 1993, Bernstein, P. et al., “Microsoft Repository Version 2 & The Open 326-333. Information Model.” Microsofi Paper pub. In Information Systems, Suzuki, H. et al., “Geometric Modeling for Modeling Products', 1999, 22(4). Proceedings of the Third international Conference on Engineering Fialliet al., “Java TMArchitecture for XML Binding (JAXB) Speci Graphics and Descriptive Geometry, Jul. 11-16, 1988, Vienna Aus fication.” Sep. 12, 2002, Version 0.7, 1-178. tria, 2, 237-243. Gordon, A.D. et al., “Typing a Multi-Language Intermediate Code.” Sreekanth, U. et al., “A Specification Environment for Configuring a POPL '01, London, UK, Jan. 2001, 257. Discrete-Part Manufacturing System Simulation Infrastructure'. Ip et al., IEEE, 1991, 8-15. International Conference on Systems, Man and Cybernetics, Oct. Lubinsky, B., "Approaches to B2B Integration.” EAI Journal, Feb. 17-20, 1993, 1,349-354. 2002. Taylor, R.H. et al., “An Integrated Robot System Architecture'. Pro Singer, J., “JVM versus CLR: A Comparative Study.” PPPJ 2003, ceedings of the IEEE, Jul. 1983, 71(7), 842-856. Kilkenny City, Ireland, Jun. 16-18, 2003, 167. Varlamis I. et al., “Bridging XML-Schema and Relational Databases. Soundarajan et al., Fifih International Conference on Sofiware reuse A System for generating and Manipulating Relational Databases ICSR, 1998, 206-215. using Valid XML Documents'. DocEng" Ol. Nov. 9-10, 2001. “Using Value Objects.” by the XDoclet Team, last published May 5, Wilcox, J., “Object Databases-Object Methods in Distributed Com 2005, http://xdoclet.sourceforge.net/xcloclet valueobjects.html. puting”. Dr. Dobbs Journal, Nov. 1994, 19(13), 26-34. Wang et al., paper on (OS) application programming interfaces Watanabe, S., “Knowledge Integration for Architectural Design”. (), IEEE Communications Magazine, Oct. 2001. Knowledge-Based Computer-Aided Architectural Design, 1994, Evans, H., "XORM.” Mapping Layer, Beta Version, Jul. 10, 2003, 19 123-146. pages, downloaded at http://sourceforge.net/projects/xorm. Waugh, A., “Specifying Metadata Standards for Metadata Tool Con McClanahan, C.R., "Apache Struts API Documentation: Class Redi figuration'. Computer Networks and ISDN Systems, 1998, 30, 23-32. rect Tag.” Version 1.1.1.1, Mar. 11, 2003, 10 pages. Wold, E. et al., “Content-Based Classification, Search, and Retrieval Modi, T., “Clean Up Your Wire Protocol with SOAP.” Java World. of Audio', IEEE Multimedia, IEEE Computer Society, 1996, 3. Apr. 27, 2001, Part 2, 15 pages. 27-36. Rizzo, T., “Introduction to Using ADO 2.5 with Microsoft Exchange Yoshikawa, M. et al., "XRel: A Path-Based Approach to Storage and 2000.” Microsoft Website, Nov. 2000, 11 pages. Retrieval of XML Documents. Using Relational Databases”. ACM Transactional On Internet technology, Aug. 2001, 1(1), 110-141. * cited by examiner U.S. Patent Jun. 22, 2010 Sheet 1 of 34 US 7,743,019 B2

19NVT

,99

????|-----*–

U.S. Patent Jun. 22, 2010 Sheet 5 of 34 US 7,743,019 B2

U.S. Patent Jun. 22, 2010 Sheet 6 of 34 US 7,743,019 B2

SseuppvE’3IOO

U.S. Patent US 7,743,019 B2

U.S. Patent Jun. 22, 2010 Sheet 9 of 34 US 7,743,019 B2 ?uOOg euuÐ?OS Suu3}|

U.S. Patent Jun. 22, 2010 Sheet 11 of 34 US 7,743,019 B2

U.S. Patent Jun. 22, 2010 Sheet 12 of 34 US 7,743,019 B2

Z?,"$DIE

U.S. Patent US 7,743,019 B2

|NEHTO

(HBEINHOSEfnS)

LNE|TO (H=Hsinand) U.S. Patent Jun. 22, 2010 Sheet 14 of 34 US 7,743,019 B2

e??CI199) e][9CI100 U.S. Patent Jun. 22, 2010 Sheet 15 of 34 US 7,743,019 B2

N V 9

O vm 9 U.S. Patent Jun. 22, 2010 Sheet 16 of 34 US 7,743,019 B2

?Uun?OA?uun?OA U.S. Patent US 7,743,019 B2

AQHV/CINTOEENIHOV/W U.S. Patent Jun. 22, 2010 Sheet 18 of 34 US 7,743,019 B2

900Z eqed Z00-Z

sesselo

TWXu! 0.1,02 (9099uo euuæ?oS U.S. Patent Jun. 22, 2010 Sheet 19 of 34 US 7,743,019 B2

UOSJ9) U.S. Patent Jun. 22, 2010 Sheet 20 of 34 US 7,743,019 B2

A)

U.S. Patent Jun. 22, 2010 Sheet 22 of 34 US 7,743,019 B2

U.S. Patent Jun. 22, 2010 Sheet 23 of 34 US 7,743,019 B2

sasselo

:3.10]Spuequello

U.S. Patent US 7,743,019 B2

GZ"SDIH U.S. Patent US 7,743,019 B2

CD c.

U.S. Patent Jun. 22, 2010 Sheet 26 of 34 US 7,743,019 B2

U.S. Patent Jun. 22 2010 Sheet 28 of 34 US 7,743,019 B2

EIGIOOONZ LINEINSDES

EGIOOISI, LNE.Wº)=IS ENCIOOCINZ ILNEINSDES

EICIOOCINZ JLNEMINS)ES

ENCIOOCINZ ·LNEWSDES \/09."SOI

U.S. Patent Jun. 22, 2010 Sheet 30 of 34 US 7,743,019 B2

EIGIOOILS!, ·LNEW50EIS ENCIOOClè18 ILNEINSOES ECIOSOCINZ 1NEINS)ES £179,"SDI-|

E?ClO3)JIS? 1.NEWSDES EKONOAICI EKOV–RIEILN|| EICIOOCINZ ILNE|W|5)ES U.S. Patent Jun. 22, 2010 Sheet 31 of 34 US 7,743,019 B2

ILNEW9BIS ENCIOOCINZ ILNEW50EIS EIGIOSOILS!, EKORJOAICI EKOVERHEILNI

EIGIOOLS|, ILNEINSOES EICIOOCINZ ·LNEW5)ES U.S. Patent Jun. 22 2010 Sheet 32 of 34 US 7,743 ,019 B2 Quæuoduo'oQNZ

((IETIc|WOOLIT”( £199"SDIE

ILNENOCHINOOCINZ U.S. Patent Jun. 22, 2010 Sheet 33 of 34 US 7,743,019 B2

ZG99

ouÁSIdV

3646'

9999 17999

Uu??SASS-HUHAA-UON)||

ZZ99 U.S. Patent Jun. 22, 2010 Sheet 34 of 34 US 7,743,019 B2

#7019

US 7,743,019 B2 1. 2 SYSTEMIS AND METHODS FOR PROVIDING U.S. patent application Ser. No. 10.693,574, filed on even SYNCHRONIZATION SERVICES FOR UNITS date herewith, entitled “SYSTEMS AND METHODS FOR OF INFORMATION MANAGEABLE BY A EXTENSIONS AND INHERITANCE FOR UNITS OF HARDWAREASOFTWARE INTERFACE INFORMATION MANAGEABLE BY A HARDWAREA SYSTEM SOFTWARE INTERFACE SYSTEM.

CROSS-REFERENCE FIELD OF THE INVENTION This application is a continuation-in-part of U.S. patent The present invention relates generally to the field of infor application Ser. No. 10/646,575, filed on Aug. 21, 2003, 10 mation storage and retrieval, and, more particularly, to an entitled SYSTEMS AND METHODS FOR INTERFAC active storage platform for organizing, searching, and sharing ING APPLICATION PROGRAMS WITH AN ITEM different types of data in a computerized system. BASED STORAGE PLATFORM, the entire contents of which are hereby incorporated herein by reference. BACKGROUND This application is related by subject matter to the inven 15 tions disclosed in the following commonly assigned applica Individual disk capacity has been growing at roughly sev tions, the contents of which are hereby incorporated into this enty percent (70%) per year over the last decade. Moore's law present application in their entirety (and partially Summa accurately predicted the tremendous gains in central process rized herein for convenience): U.S. patent application Ser. ing unit (CPU) power that has occurred over the years. Wired No. 10/647,058, filed on Aug. 21, 2003, entitled “SYSTEMS and wireless technologies have provided tremendous connec AND METHODS FOR REPRESENTING UNITS OF tivity and bandwidth. Presuming current trends continue, INFORMATION MANAGEABLE BY A HARDWAREA within several years the average laptop computer will possess SOFTWARE INTERFACE SYSTEM BUT INDEPEN roughly one terabyte (TB) of storage and contain millions of DENT OF PHYSICAL REPRESENTATION”; U.S. patent files, and 500 gigabyte (GB) drives will become common application Ser. No. 10/646,941, filed on Aug. 21, 2003, 25 place. entitled “SYSTEMS AND METHODS FOR SEPARATING Consumers use their computers primarily for communica UNITS OF INFORMATION MANAGEABLE BY A tion and organizing personal information, whether it is tradi HARDWAREASOFTWARE INTERFACE SYSTEM FROM tional personal information manager (PIM) style data or THEIR PHYSICAL ORGANIZATION”; U.S. patent appli media Such as digital music or photographs. The amount of cation Ser. No. 10/646,940, filed on Aug. 21, 2003, entitled 30 digital content, and the ability to store the raw , has SYSTEMS AND METHODS FOR THE IMPLEMENTA increased tremendously; however the methods available to TION OFA BASE SCHEMA FOR ORGANIZING UNITS consumers for organizing and unifying this data has not kept OF INFORMATION MANAGEABLE BYAHARDWAREA pace. Knowledge workers spend enormous amounts of time SOFTWARE INTERFACE SYSTEM: U.S. patent applica managing and sharing information, and some studies esti tion Ser. No. 10/646,632, filed on Aug. 21, 2003, entitled 35 mate that knowledge workers spend 15-25% of their time on SYSTEMS AND METHODS FOR THE IMPLEMENTA non-productive information related activities. Other studies TION OF A CORE SCHEMA FOR PROVIDING A TOP estimate that a typical knowledge worker spends about 2.5 LEVEL STRUCTURE FOR ORGANIZING UNITS OF hours per day searching for information. INFORMATION MANAGEABLE BY A HARDWAREA Developers and information technology (IT) departments SOFTWARE INTERFACE SYSTEM: U.S. patent applica 40 invest significantamounts of time and money in building their tion Ser. No. 10/646,645, filed on Aug. 21, 2003, entitled own data stores for common storage abstractions to represent SYSTEMS AND METHOD FOR REPRESENTING Such things as , places, times, and events. Not only RELATIONSHIPS BETWEEN UNITS OF INFORMA does this result in duplicated work, but it also creates islands TION MANAGEABLE BY A HARDWAREASOFTWARE of common data with no mechanisms for common searching INTERFACE SYSTEM”; U.S. patent application Ser. No. 45 or sharing of that data. Just consider how many address books 10/646,646, filed on Aug. 21, 2003, entitled “STORAGE can exist today on a computer running the Microsoft Win PLATFORM FOR ORGANIZING, SEARCHING, AND dows . Many applications, such as e- SHARING DATA'; U.S. patent application Ser. No. 10/646, clients and personal finance programs, keep individual 580, filed on Aug. 21, 2003, entitled “SYSTEMS AND address books, and there is little sharing among applications METHODS FOR DATA MODELING IN AN ITEM 50 of the address book data that each Such program individually BASED STORAGE PLATFORM: U.S. patent application maintains. Consequently, a finance program (like Microsoft Ser. No. 10,692,779, filed on even date herewith, entitled Money) does not share addresses for payees with the SYSTEMS AND METHODS FOR THE IMPLEMENTA addresses maintained in an email contact folder (like the one TION OF A DIGITAL IMAGES SCHEMA FOR ORGA in ). Indeed, many users have multiple NIZING UNITS OF INFORMATION MANAGEABLE BY 55 devices and logically should synchronize their personal data A HARDWARE/SOFTWARE INTERFACE SYSTEM: amongst themselves and across a wide variety of additional U.S. patent application Ser. No. 10,692,508, filed on even Sources, including cellphones to commercial services such as date herewith, entitled “SYSTEMS AND METHODS FOR MSN and AOL: nevertheless, collaboration of shared docu PROVIDING RELATIONAL AND HIERARCHICAL ments is largely achieved by attaching documents to e-mail SYNCHRONIZATION SERVICES FOR UNITS OF 60 messages—that is, manually and inefficiently. INFORMATION MANAGEABLE BY A HARDWAREA One reason for this lack of collaboration is that traditional SOFTWARE INTERFACE SYSTEM: U.S. patent applica approaches to the organization of information in computer tion Ser. No. 10,693,362, filed on even date herewith, entitled systems have centered on the use of file-folder-and-directory SYSTEMS AND METHODS FOR THE IMPLEMENTA based systems (“file systems) to organize pluralities of files TION OF A SYNCHRONIZATION SCHEMAS FOR 65 into directory hierarchies offolders based on an abstraction of UNITS OF INFORMATION MANAGEABLE BY A the physical organization of the storage medium used to store HARDWARE/SOFTWARE INTERFACE SYSTEM”; and the files. The Multics operating system, developed during the US 7,743,019 B2 3 4 1960s, can be credited with pioneering the use of the files, based hierarchical structure at the hardware/software inter folders, and directories to manage storable units of data at the face system level. Other efforts, such as those that attempted operating system level. Specifically, Multics used symbolic to use SmallTalk (and other derivatives), proved to be quite addresses within a hierarchy of files (thereby introducing the effective at handling file and non-file representations but idea of a file path) where physical addresses of the files were not transparent to the user (applications and end-users). This lacked database features necessary to efficiently organize and file system was entirely unconcerned with the of utilize the relationships that exist between the various data any individual file, and the relationships amongst and files, and thus the overall efficiency of such systems was between files was deemed irrelevant at the operating system unacceptable. Yet other attempts to use BeOS (and other such level (that is, other than the location of the file within the 10 operating systems research) proved to be inadequate at han hierarchy). Since the advent of Multics, storable data has been dling non-file representations—the same core shortcoming of organized into files, folders, and directories at the operating traditional file systems—despite being able to adequately system level. These files generally include the file hierarchy represent files while providing some necessary database fea itself (the “directory') embodied in a special file maintained tures. by the file system. This directory, in turn, maintains a list of 15 entries corresponding to all of the other files in the directory Database technology is another area of the art in which and the nodal location of such files in the hierarchy (herein similar challenges exits. For example, while the relational referred to as the folders). Such has been the state of the art for database model has been a great commercial Success, in truth approximately forty years. independent software vendors (ISV) generally exercise a However, while providing a reasonable representation of small portion of the functionality available in relational data information residing in the computer's physical storage sys base software products (such as Microsoft SQL Server). tem, a file system is nevertheless an abstraction of that physi Instead, most of an applications interaction with Such a prod cal storage system, and therefore utilization of the files uct is in the form of simple “gets” and “puts”. While there are requires a level of indirection (interpretation) between what a number of readily apparent reasons for this—such as being the user manipulates (units having context, features, and rela 25 tionships to other units) and what the operating system pro platform or database agnostic—one key reason that often vides (files, folders, and directories). Consequently, users goes unnoticed is that the database does not necessarily pro (applications and/or end-users) have no choice but to force vide the exact abstractions that a major business application units of information into a file system structure even when vendor really needs. For example, while the real world has the doing so is inefficient, inconsistent, or otherwise undesirable. 30 notion of “items, such as “customers' or “orders' (along Moreover, existing file systems know little about the structure with an order's embedded “line items' as items in and of of data stored in individual files and, because of this, most of themselves), relational databases only talk in terms of tables the information remains locked up in files that may only be and rows. Consequently, while the application may desire to accessed (and comprehensible) to the applications that wrote have aspects of consistency, locking, security, and/or triggers them. Consequently, this lack of Schematic description of 35 at the item level (to name a few), generally databases provide information, and mechanisms for managing information, these features only at the table/row level. While this may work leads to the creation of silos of data with little data sharing fine if each item gets mapped to a single row in some table in among the individual silos. For example, many personal com the database, in the case of an order with multiple line items puter (PC) users have more than five distinct stores that con there may be reasons why an item actually gets mapped to tain information about the people they interact with on some 40 level—for example, Outlook Contacts, online account multiple tables and, when that is the case, the simple rela addressees, , Quicken Payees, and tional database system does not quite provide the right instant messaging (IM) buddy lists—because organizing files abstractions. Consequently, an application must build logic presents a significant challenge to these PC users. Because on top of the database to provide these basic abstractions. In most existing file systems utilize a nested folder metaphor for 45 other words, the basic does not provide a organizing files and folders, as the number of files increases sufficient platform for storage of data on which higher-level the effort necessary to maintain an organization scheme that applications can easily be developed because the basic rela is flexible and efficient becomes quite daunting. In such situ tional model requires a level of indirection between the appli ations, it would be very useful to have multiple classifications cation and the storage system—where the semantic structure of a single file; however, using hard or soft links in existing 50 of the data might only be visible in the application in certain file systems is cumbersome and difficult to maintain. instances. While Some database vendors are building higher Several unsuccessful attempts to address the shortcomings level functionality into their products—such as providing of file systems have been made in the past. Some of these object relational capabilities, new organizational models, and previous attempts have involved the use of content address the like—none have yet to provide the kind of comprehensive able memory to provide a mechanism whereby data could be 55 Solution needed, where a truly comprehensive solution is one accessed by content rather than by physical address. How which provides both useful data model abstractions (such as ever, these efforts have proven unsuccessful because, while “Items.” “Extensions.” “Relationships, and so on) for useful content addressable memory has proven useful for Small domain abstractions (such as “Persons.” “Locations.” scale use by devices such as caches and memory management “Events.” etc.). units, large-scale use for devices such as physical storage 60 media has not yet been possible for a variety of reasons, and In of the foregoing deficiencies in existing data stor thus such a solution simply does not exist. Other attempts age and database technologies, there is a need for a new using object-oriented database (OODB) systems have been storage platform that provides an improved ability to orga made, but these attempts, while featuring strong database nize, search, and share all types of data in a computer sys characteristics and good non-file representations, were not 65 tem—a storage platform that extends and broadens the data effective in handling file representations and could not repli platform beyond existing file systems and database systems, cate the speed, efficiency, and simplicity of the file and folder and that is designed to be the store for all types of data. The US 7,743,019 B2 5 6 present invention, together with the related inventions incor API) of the storage platform—for example, an Item might be porated by reference earlier herein, satisfies this need. the current location of a cellphone or the temperature reading on a temperature sensor. The hardware/software interface SUMMARY system may manipulate a plurality of Items, and may further comprise Items interconnected by a plurality of Relationships The following summary provides an overview of various managed by the hardware/software interface system. aspects of the invention described in the context of the related inventions incorporated-by-reference earlier herein (the A hardware/software interface system for the computer “related inventions'). This summary is not intended to pro system further comprises a core schema to define a set of core vide an exhaustive description of all of the important aspects 10 Items which said hardware/software interface system under of the invention, nor to define the scope of the invention. stands and can directly process in a predetermined and pre Rather, this Summary is intended to serve as an introduction to dictable way. To manipulate a plurality of Items, the computer the detailed description and figures that follow. system interconnects said Items with a plurality of Relation The present invention, as well as the related inventions, are ships and manages said Relationships at the hardware/soft collectively directed to a storage platform for organizing, 15 ware interface system level. searching, and sharing data. The storage platform of the The API of the storage platform provides data classes for present invention extends and broadens the concept of data each item, item extension, and relationship defined in the set storage beyond existing file systems and database systems, of storage platform schemas. In addition, the application pro and is designed to be the store for all types of data including gramming interface provides a set of framework classes that structured, non-structured, or semi-structured data. define a common set of behaviors for the data classes and that, The storage platform of the present invention comprises a together with the data classes, provide the basic programming data store implemented on a database engine. The database model for the storage platform API. The storage platform API engine comprises a relational database engine with object provides a simplified query model that enables application relational extensions. The data store implements a data model programmers to form queries based on various properties of that Supports organization, searching, sharing, synchroniza 25 the items in the data store, in a manner that insulates the tion, and security of data. Specific types of data are described application programmer from the details of the query lan in schemas, and the platform provides a mechanism to extend guage of the underlying database engine. The storage plat the set of schemas to define new types of data (essentially form API also collects changes to an item made by an appli Subtypes of the basic types provides by the schemas). A cation program and then organizes them into the correct synchronization capability facilitates the sharing of data 30 updates required by the database engine (or any kind of Stor among users or systems. File-system-like capabilities are age engine) on which the data store is implemented. This provided that allow interoperability of the data store with enables application programmers to make changes to an item existing file systems but without the limitation of such tradi in memory, while leaving the complexity of data store updates tional file systems. A change tracking mechanism provides to the API. the ability track changes to the data store. The storage plat 35 Through its common storage foundation and schematized form further comprises a set of application program inter data, the storage platform of the present invention enables faces that enable applications to access all of the foregoing more efficient application development for consumers, capabilities of the storage platform and to access the data knowledge workers and enterprises. It offers a rich and exten described in the schemas. sible application programming interface that not only makes The data model implemented by the data store defines units 40 available the capabilities inherent in its data model, but also of data storage in terms of items, elements, and relationships. embraces and extends existing file system and database An item is a unit of data storable in a data store and can access methods. comprise one or more elements and relationships. An element As part of this overarching structure of interrelated inven is an instance of a type comprising one or more fields (also tions (described in detail in Section II of the Detailed Descrip referred to herein as a property). A relationship is a link 45 tion), the present invention is specifically directed to the between two items. (As used herein, these and other specific Synchronization APIs (described in detail in Section III of the terms may be capitalized in order to offset them from other Detailed Description). Other features and advantages of the terms used in close proximity; however, there is no intention invention may become apparent from the following detailed whatsoever to distinguish between a capitalized term, e.g. description of the invention and accompanying drawings. “Item', and the same term when not capitalized, e.g., “item'. 50 and no such distinction should be presumed or implied.) The computer system further comprises a plurality of Items BRIEF DESCRIPTION OF THE DRAWINGS where each Item constitutes a discrete storable unit of infor mation that can be manipulated by a hardware/software inter The foregoing Summary, as well as the following detailed face system; a plurality of Item Folders that constitute an 55 description of the invention, is better understood when read in organizational structure for said Items; and a hardware/soft conjunction with the appended drawings. For the purpose of ware interface system for manipulating a plurality of Items illustrating the invention, there is shown in the drawings and wherein each Item belongs to at least one Item Folder and exemplary embodiments of various aspects of the invention; may belong to more than one Item Folder. however, the invention is not limited to the specific methods An Item or some of the Items property values may be 60 and instrumentalities disclosed. In the drawings: computed dynamically as opposed to being derived from a FIG. 1 is a block diagram representing a computer system persistent store. In other words, the hardware/software inter in which aspects of the present invention may be incorpo face system does not require that the Item be stored, and rated; certain operations are Supported Such as the ability to enu FIG. 2 is a block diagram illustrating a computer system merate the current set of Items or the ability to retrieve an Item 65 divided into three component groups: the hardware compo given its identifier (which is more fully described in the sec nent, the hardware/software interface system component, and tions that describe the application programming interface, or the application programs component; US 7,743,019 B2 7 8 FIG. 2A illustrates the traditional tree-based hierarchical FIG. 28 is a diagram illustrating the concept of an Item structure for files grouped in folders in a directory in a file search view; based operating system; FIG. 29 is a diagram illustrating an exemplary Item hier FIG. 3 is a block diagram illustrating a storage platform; archy; FIG. 4 illustrates the structural relationship between Items, FIG. 30A illustrates an interface Interfacel as a conduit Item Folders, and Categories; through which first and second code segments communicate; FIG. 5A is a block diagram illustrating the structure of an FIG. 30B illustrates an interface as comprising interface Item; objects I1 and I2 which enable first and second code segments FIG. 5B is a block diagram illustrating the complex prop of a system to communicate via medium M: erty types of the Item of FIG. 5A; 10 FIG.31A illustrates how the function provided by interface FIG.5C is a block diagram illustrating the “Location’ Item Interfacel may be subdivided to convert the communications wherein its complex types are further described (explicitly of the interface into multiple interfaces Interface1A, Interface listed): 1B, Interface 1C: FIG. 6A illustrates an Item as a subtype of the Item found FIG.31B illustrates how the function provided by interface in the Base Schema: 15 I1 may be subdivided into multiple interfaces I1a, I1b, I1c. FIG. 6B is a block diagram illustrating the subtype Item of FIG. 32A illustrates a scenario where a meaningless FIG. 6A wherein its inherited types are explicitly listed (in parameter precision can be ignored or replaced with an arbi addition to its immediate properties); trary parameter; FIG. 7 is a block diagram illustrating the Base Schema FIG. 32B illustrates a scenario where an interface is including its two top-level class types, Item and Property replaced by a substitute interface that is defined to ignore or Base, and the additional Base Schema types derived there add parameters to an interface; from; FIG. 33A illustrates a scenario where a 1st and 2nd Code FIG. 8A is a block diagram illustrating Items in the Core Segments are merged into a module containing them both; Schema: FIG. 33B illustrates a scenario where part or all of an FIG.8B is a block diagram illustrating the property types in 25 interface may be written inline into another interface to form the Core Schema: a merged interface. FIG. 9 is a block diagram illustrating an Item Folder, its FIG.34A illustrates how one or more pieces of middleware member Items, and the interconnecting Relationships might convert communications on the first interface to con between the Item Folder and its member Items; form them to one or more different interfaces: FIG. 10 is a block diagram illustrating a Category (which, 30 FIG.34B illustrates how a code segment can be introduced again, is an Item itself), its member Items, and the intercon with an interface to receive the communications from one necting Relationships between the Category and its member interface but transmit the functionality to second and third Items; interfaces: FIG. 11 is a diagram illustrating a reference type hierarchy FIG. 35A illustrates how a just-in-time compiler (JIT) of the data model of the storage platform: 35 might convert communications from one code segment to FIG. 12 is a diagram illustrating how relationships are another code segment; classified; FIG.35B illustrates a JIT method of dynamically rewriting FIG. 13 is a diagram illustrating a notification mechanism; one or more interfaces may be applied to dynamically factor FIG. 14 is a diagram illustrating an example in which two or otherwise alter said interface; transactions are both inserting a new record into the same 40 FIG. 36 illustrates a three instances of a common data store B-Tree; and the components for synchronizing them; and FIG. 15 illustrates a data change detection process; FIG. 37 illustrates one embodiment of the present inven FIG. 16 illustrates an exemplary directory tree; tion that presumes a simple adapter that is unaware of how FIG. 17 shows an example in which an existing folder of a state is calculated or its associated metadata is exchanged. directory-based file system is moved into the storage platform 45 data store; DETAILED DESCRIPTION FIG. 18 illustrates the concept of Containment Folders; FIG. 19 illustrates the basic architecture of the storage platform API: FIG. 20 schematically represents the various components 50 of the storage platform API stack; TABLE OF CONTENTS FIG. 21A is a pictorial representation of an exemplary I. INTRODUCTION 18 A. EXEMPLARY COMPUTING ENVIRONMENT 18 Contacts Item schema: B. TRADITIONAL FILE-BASED STORAGE 22 FIG. 21B is a pictorial representation of the Elements for 55 II. WINFS STORAGE PLATFORM FOR ORGANIZING, 24 the exemplary Contacts Item schema of FIG. 21A: SEARCHING, AND SHARING DATA FIG. 22 illustrates the runtime framework of the storage A. GLOSSARY 24 platform API: B. STORAGE PLATFORMOVERVIEW 25 FIG. 23 illustrates the execution of a “FindAll operation; C. THE DATA MODEL 26 1. Items 28 FIG. 24 illustrates the process by which storage platform 60 2. Item Identification 32 API classes are generated from the storage platform Schema: 3. Item Folders and Categories 32 FIG. 25 illustrates a schema on which a File API is based; 4 Schemas 34 FIG. 26 is a diagram illustrating an access mask format a) Base Schema 34 b) Core Schema 35 used for data security purposes; 5. Relationships 37 FIG. 27 (parts a, b, and c) depicts a new identically pro 65 a) Relationship Declaration 38 tected security region being carved out of an existing security b) Holding Relationship 39 region; US 7,743,019 B2 9 10

-continued -continued

TABLE OF CONTENTS TABLE OF CONTENTS c) Embedding Relationships 40 d) Conflict Handling 87 d) Reference Relationships 41 (1) Conflict Detection 87 e) Rules and Constraints 42 (a) Knowledge-Based 87 f) Ordering of Relationships 42 Conflicts 6. Extensibility 48 (b) Constraint-Based 88 a) Item extensions 49 Conflicts b) Extending NestedElement 53 10 (2) Conflict Processing 88 types (a) Automatic Conflict 89 D. DATABASE ENGINE 55 Resolution 1. Data Store Implementation 56 (b) Conflict Logging 89 Using UDTs (c) Conflict Inspection 90 Item Mapping 57 and Resolution Extension Mapping 58 15 (d) Convergence of Replicas 90 Nested Element Mapping 58 and Propagation of Object Identity 58 Conflict Resolutions SQL Object Naming 58 2. Synchronizing to Non-Storage 91 Column Naming 59 Platform Data Stores Search Views 60 a) Sync Services 91 a) Item 61 (1) Change Enumeration 91 (1) Master Item 61 (2) Change Application 92 Search View (3) Conflict Resolution 93 (2) Typed Item 61 b) Adapter Implementation 93 Search Views 3. Security 94 b) Item Extensions 62 4. Manageability 94 (1) Master Extension 62 B. SYNCHRONIZATIONAPI OVERVIEW 95 Search View 25 1. General Terminology 95 (2) Typed Extension 63 2. Synchronization API Principals 97 Search Views C. SYNCHRONIZATIONAPI SERVICES 99 c) Nested Elements 63 1. Change Enumeration 99 d) Relationships 64 2. Change Application 1OO (1) Master Relationship 64 3. Sample Code 1OO Search View 30 4. Methods of API Synchronization 104 64 D ADDITIONALASPECTS OF THE SYNC SCHEMA 106 (2) Relationship Instance IV. CONCLUSION 108 Search Views e) 65 9. Updates 65 10. Change Tracking & Tombstones 66 35 I. Introduction a) Change Tracking 66 (1) Change Tracking in 66 The subject matter of the present invention is described “Master SearchViews with specificity to meet statutory requirements. However, the (2) Change Tracking in 67 description itself is not intended to limit the scope of this “Typed Search Views patent. Rather, the inventors have contemplated that the b) Tombstones 68 40 claimed Subject matter might also be embodied in other ways, (1) Item Tombstones 68 to include different steps or combinations of steps similar to (2) Extension Tombstones 69 (3) Relationships Tombstone 69 the ones described in this document, in conjunction with other (4) Tombstone Cleanup 70 present or future technologies. Moreover, although the term 11. Helper APIs and Functions 70 “step” may be used herein to connote different elements of a) Function 70 45 methods employed, the term should not be interpreted as System.Storage...GetItem implying any particular order among or between various steps b) Function 70 System.Storage...GetExtension herein disclosed unless and except when the order of indi c) Function 71 vidual steps is explicitly described. System.Storage...GetRelationship A. Exemplary Computing Environment 12. Metadata 71 50 Numerous embodiments of the present invention may a) Schema Metadata 71 b) Instance Metadata 71 execute on a computer. FIG. 1 and the following discussion is E. SECURITY 71 intended to provide a brief general description of a suitable F. NOTIFICATIONS AND CHANGE TRACKING 72 computing environment in which the invention may be imple G. TRADITIONAL FILE SYSTEM 73 mented. Although not required, various aspects of the inven INTEROPERABILITY 55 tion may be described in the general context of computer H. STORAGE PLATFORM API 74 executable instructions, such as program modules, being III. SYNCHRONIZATIONAPI 8O A. SYNCHRONIZATION OVERVIEW 81 executed by a computer, Such as a client workstation or a 1. Storage-Platform-to-Storage 81 server. Generally, program modules include routines, pro Platform Synchronization grams, objects, components, data structures and the like that a) Synchronization (Sync) 82 60 perform particular tasks or implement particular abstract data Controlling Applications types. Moreover, the invention may be practiced with other b) Schema annotation 83 computer system configurations, including hand held c) Sync Configuration 84 (1) Community Folder 85 devices, multi processor systems, microprocessor based or Mappings programmable consumer electronics, network PCs, mini (2) Profiles 86 65 computers, mainframe computers and the like. The invention (3) Schedules 87 may also be practiced in distributed computing environments where tasks are performed by remote processing devices that US 7,743,019 B2 11 12 are linked through a communications network. In a distrib above relative to the personal computer 20, although only a uted computing environment, program modules may be memory storage device 50 has been illustrated in FIG.1. The located in both local and remote memory storage devices. logical connections depicted in FIG. 1 include a local area As shown in FIG. 1, an exemplary general purpose com network (LAN)51 and a wide area network (WAN)52. Such puting system includes a conventional personal computer 20 networking environments are commonplace in offices, enter or the like, including a processing unit 21, a system memory prise wide computer networks, intranets and the Internet. 22, and a system buS 23 that couples various system compo When used in a LAN networking environment, the per nents including the system memory to the processing unit 21. sonal computer 20 is connected to the LAN 51 through a The system bus 23 may be any of several types of bus struc network interface or adapter 53. When used in a WAN net tures including a memory bus or memory controller, a periph 10 working environment, the personal computer 20 typically eral bus, and a local bus using any of a variety of bus archi includes a modem 54 or other means for establishing com tectures. The system memory includes read only memory munications over the wide area network52, such as the Inter (ROM) 24 and random access memory (RAM) 25. A basic net. The modem 54, which may be internal or external, is input/output system 26 (BIOS), containing the basic routines connected to the system bus 23 via the serial port interface 46. that help to transfer information between elements within the 15 In a networked environment, program modules depicted rela personal computer 20, Such as during start up, is stored in tive to the personal computer 20, or portions thereof, may be ROM 24. The personal computer 20 may further include a stored in the remote memory storage device. It will be appre hard disk drive 27 for reading from and writing to a hard disk, ciated that the network connections shown are exemplary and not shown, a magnetic disk drive 28 for reading from or other means of establishing a communications link between writing to a removable magnetic disk 29, and an optical disk the computers may be used. drive 30 for reading from or writing to a removable optical As illustrated in the block diagram of FIG. 2, a computer disk 31 such as a CD ROM or other optical media. The hard system 200 can be roughly divided into three component disk drive 27, magnetic disk drive 28, and optical disk drive groups: the hardware component 202, the hardware/software 30 are connected to the system bus 23 by a hard disk drive interface system component 204, and the applications pro interface 32, a magnetic disk drive interface 33, and an optical 25 grams component 206 (also referred to as the “user compo drive interface 34, respectively. The drives and their associ nent' or “software component” in certain contexts herein). ated computer readable media provide non Volatile storage of In various embodiments of a computer system 200, and computer readable instructions, data structures, program referring back to FIG. 1, the hardware component 202 may modules and other data for the personal computer 20. comprise the central processing unit (CPU) 21, the memory Although the exemplary environment described herein 30 (both ROM 24 and RAM 25), the basic input/output system employs a hard disk, a removable magnetic disk 29 and a (BIOS) 26, and various input/output (I/O) devices such as a removable optical disk 31, it should be appreciated by those keyboard 40, a mouse 42, a monitor 47, and/or a printer (not skilled in the art that other types of computer readable media shown), among other things. The hardware component 202 which can store data that is accessible by a computer, such as comprises the basic physical infrastructure for the computer magnetic cassettes, cards, digital video disks, 35 system 200. Bernoulli cartridges, random access memories (RAMS), read The applications programs component 206 comprises vari only memories (ROMs) and the like may also be used in the ous Software programs including but not limited to compilers, exemplary operating environment. Likewise, the exemplary database systems, word processors, business programs, environment may also include many types of monitoring Videogames, and so forth. Application programs provide the devices Such as heat sensors and security or fire alarm sys 40 tems, and other sources of information. means by which computer resources are utilized to Solve A number of program modules may be stored on the hard problems, provide Solutions, and process data for various disk, magnetic disk 29, optical disk 31, ROM 24 or RAM 25, users (machines, other computer systems, and/or end-users). including an operating system 35, one or more application The hardware/software interface system component 204 programs 36, other program modules 37 and program data 38. 45 comprises (and, in some embodiments, may solely consist of) A user may enter commands and information into the per an operating system that itself comprises, in most cases, a Sonal computer 20 through input devices such as a keyboard shell and a kernel. An “operating system” (OS) is a special 40 and pointing device 42. Other input devices (not shown) program that acts as an intermediary between application may include a microphone, joystick, game pad, satellite disk, programs and computer hardware. The hardware/software scanner or the like. These and other input devices are often 50 interface system component 204 may also comprise a virtual connected to the processing unit 21 through a serial port machine manager (VMM), a Common Language Runtime interface 46 that is coupled to the system bus, but may be (CLR) or its functional equivalent, a Java Virtual Machine connected by other interfaces, such as a parallel port, game (JVM) or its functional equivalent, or other such software port or universal serial bus (USB). A monitor 47 or other type components in the place of or in addition to the operating of display device is also connected to the system bus 23 via an 55 system in a computer system. The purpose of a hardware/ interface. Such as a video adapter 48. In addition to the moni Software interface system is to provide an environment in tor 47, personal computers typically include other peripheral which a user can execute application programs. The goal of output devices (not shown). Such as speakers and printers. any hardware/software interface system is to make the com The exemplary system of FIG. 1 also includes a host adapter puter system convenient to use, as well as utilize the computer 55, Small Computer System Interface (SCSI) bus 56, and an 60 hardware in an efficient manner. external storage device 62 connected to the SCSI bus 56. The hardware/software interface system is generally The personal computer 20 may operate in a networked loaded into a computer system at startup and thereafter man environment using logical connections to one or more remote ages all of the application programs in the computer system. computers, such as a remote computer 49. The remote com The application programs interact with the hardware/soft puter 49 may be another personal computer, a server, a router, 65 ware interface system by requesting services via an applica a network PC, a peer device or other common network node, tion program interface (API). Some application programs and typically includes many or all of the elements described enable end-users to interact with the hardware/software inter US 7,743,019 B2 13 14 face system via a user interface such as a command language equivalent terms for and references to hierarchical informa or a (GUI). tion storage structures and their folder and file components. A hardware/software interface system traditionally per Traditionally, a directory (a.k.a. a directory of folders) is a forms a variety of services for applications. In a multitasking tree-based hierarchical structure wherein files are grouped hardware/software interface system where multiple programs into folders and folder, in turn, are arranged according to may be running at the same time, the hardware/software relative nodal locations that comprise the directory tree. For interface system determines which applications should run in example, as illustrated in FIG. 2A, a DOS-based file system what order and how much time should be allowed for each base folder (or “root directory') 212 may comprise a plurality application before Switching to another application for a turn. offolders 214, each of which may further comprise additional The hardware/software interface system also manages the 10 folders (as “subfolders' of that particular folder) 216, and sharing of internal memory among multiple applications, and each of these may also comprise additional folders 218 ad handles input and output to and from attached hardware infinitum. Each of these folders may have one or more files devices such as hard disks, printers, and dial-up ports. The 220 although, at the hardware/software interface system hardware/software interface system also sends messages to level, the individual files in a folder have nothing in common each application (and, in certain case, to the end-user) regard 15 other than their location in the tree hierarchy. Not surpris ing the status of operations and any errors that may have ingly, this approach of organizing files into folder hierarchies occurred. The hardware/software interface system can also indirectly reflects the physical organization of typical storage offload the management of batch jobs (e.g., printing) so that media used to store these files (e.g., hard disks, floppy disks, the initiating application is freed from this work and can CD-ROMs, etc.). resume other processing and/or operations. On computers In addition to the foregoing, each folder is a container for that can provide parallel processing, a hardware/software its subfolders and its files—that is, each folder owns its sub interface system also manages dividing a program so that it folders and files. For example, when a folder is deleted by the runs on more than one processor at a time. hardware/software interface system, that folder's subfolders A hardware/software interface system shell (simply and files are also deleted (which, in the case of each subfolder, referred to herein as a “shell’) is an interactive end-user 25 further includes its own subfolders and files recursively). interface to a hardware/software interface system. (A shell Likewise, each file is generally owned by only one folder and, may also be referred to as a "command interpreter” or, in an although a file can be copied and the copy located in a differ operating system, as an “operating system shell’). A shell is ent folder, a copy of a file is itself a distinct and separate unit the outer layer of a hardware/software interface system that is that has no direct connection to the original (e.g., changes to directly accessible by application programs and/or end-users. 30 the original file are not mirrored in the copy file at the hard In contrast to a shell, a kernel is a hardware/software interface ware/software interface system level). In this regard, files and systems innermost layer that interacts directly with the hard folders are therefore characteristically “physical in nature ware components. because folders are the treated like physical containers, and While it is envisioned that numerous embodiments of the files are treated as discrete and separate physical elements present invention are particularly well-suited for computer 35 inside these containers. ized systems, nothing in this document is intended to limit the II. Winfs Storage Platform for Organizing, Searching, and invention to such embodiments. On the contrary, as used Sharing Data herein the term “computer system’ is intended to encompass The present invention, in combination with the related any and all devices capable of storing and processing infor inventions incorporated by reference as discussed earlier mation and/or capable of using the stored information to 40 herein, is directed to a storage platform for organizing, control the behavior or execution of the device itself, regard searching, and sharing data. The storage platform of the less of whether such devices are electronic, mechanical, logi present invention extends and broadens the data platform cal, or virtual in nature. beyond the kinds of existing file systems and database sys B. Traditional File-based Storage 45 tems discussed above, and is designed to be the store for all In most computer systems today, “files' are units of Stor types of data, including a new form of data called Items. able information that may include the hardware/software A. Glossary interface system as well as application programs, data sets, As used herein and in the claims, the following terms have and so forth. In all modern hardware/software interface sys the following meanings: tems (Windows, Unix, Linux, Mac OS, virtual machine sys 50 An "Item' is an unit of storable information accessible to a tems, and so forth), files are the basic discrete (storable and hardware/software interface system that, unlike a simple retrievable) units of information (e.g., data, programs, and so file, is an object having a basic set of properties that are forth) that can be manipulated by the hardware/software commonly supported across all objects exposed to an interface system. Groups of files are generally organized in end-user by the hardware/software interface system “folders. In , the Macintosh OS, and 55 shell. Items also have properties and relationships that other hardware/software interface systems, a folder is a col are commonly Supported across all Item types including lection of files that can be retrieved, moved, and otherwise features that allow new properties and relationships to be manipulated as single units of information. These folders, in introduced (and discussed in great detail later herein). turn, are organized in a tree-based hierarchical arrangement An "operating system” (OS) is a special program that acts called a “directory’ (discussed in more detail herein below). 60 as an intermediary between application programs and In certain other hardware/software interface systems, such as computer hardware. An operating system comprises, in DOS, Z/OS and most Unix-based operating systems, the most cases, a shell and a kernel. terms “directory” and/or “folder are interchangeable, and A “hardware/software interface system” is software, or a early Apple computer systems (e.g., the Apple IIe) used the combination of hardware and software, that serves as the term “catalog instead of directory; however, as used herein, 65 interface between the underlying hardware components all of these terms are deemed to be synonymous and inter of a computer system and applications that execute on changeable and are intended to further include all other the computer system. A hardware/software interface US 7,743,019 B2 15 16 system typically comprises (and, in some embodiments, operating system, a virtual machine manager (VMM), a may solely consist of) an operating system. A hardware/ Common Language Runtime (CLR) or its functional equiva Software interface system may also comprise a virtual lent, or a Java Virtual Machine (JVM) or its functional equiva machine manager (VMM), a Common Language Runt lent. Through its common storage foundation, and schema ime (CLR) or its functional equivalent, a Java Virtual 5 tized data, the storage platform of the present invention Machine (JVM) or its functional equivalent, or other enables more efficient application development for consum Such software components in the place of or in addition ers, knowledge workers and enterprises. It offers a rich and to the operating system in a computer system. The pur extensible programming Surface area that not only makes pose of a hardware/software interface system is to pro available the capabilities inherent in its data model, but also vide an environment in which a user can execute appli 10 embraces and extends existing file system and database cation programs. The goal of any hardware/software access methods. interface system is to make the computer system conve In the following description, and in various ones of the nient to use, as well as utilize the computer hardware in figures, the storage platform 300 of the present invention may an efficient manner. be referred to as “WinFS. However, use of this name to refer B. Storage Platform Overview 15 to the storage platform is solely for convenience of descrip Referring to FIG. 3, a storage platform 300 comprises a tion and is not intended to be limiting in any way. data store 302 implemented on a database engine 314. In one C. The Data Model embodiment, the database engine comprises a relational data The data store 302 of the storage platform 300 of the base engine with object relational extensions. In one embodi present invention implements a data model that Supports the ment, the relational database engine 314 comprises the organization, searching, sharing, Synchronization, and Secu Microsoft SQL Server relational database engine. The data rity of data that resides in the store. In the data model of the store 302 implements a data model 304 that supports the present invention, an “Item' is the fundamental unit of stor organization, searching, sharing, Synchronization, and Secu age information. The data model provides a mechanism for rity of data. Specific types of data are described in Schemas, declaring Items and Item extensions and forestablishing rela such as schemas 340, and the storage platform 300 provides 25 tools 346 for deploying those schemas as well as for extend tionships between Items and for organizing Items in Item ing those schemas, as described more fully below. Folders and in Categories, as described more fully below. A change tracking mechanism 306 implemented within the The data model relies on two primitive mechanisms, Types data store 302 provides the ability track changes to the data and Relationships. Types are structures which provide a for store. The data store 302 also provides security capabilities 30 mat which governs the form of an instance of the Type. The 308 and a promotion/demotion capability 310, both of which format is expressed as an ordered set of Properties. A Property are discussed more fully below. The data store 302 also pro is a name for a value or set of values of a given Type. For vides a set of application programming interfaces 312 to example a USPostalAddress type might have the properties expose the capabilities of the data store 302 to other storage Street, City, Zip, State in which Street, City and State are of platform components and application programs (e.g., appli 35 type String and Zip is of Type Int32. Street may be multi cation programs 350a,350b, and 350c) that utilize the storage valued (i.e. a set of values) allowing the address to have more platform. The storage platform of the present invention still than one value for the Street property. The system defines further comprises an application programming interfaces certain primitive types that can be used in the construction of (API) 322, which enables application programs, such as other types—these include String, Binary, Boolean, Int16, application programs 350a, 350b, and 350c, to access all of 40 Int32, Intó4, Single, Double, , DateTime, Decimal and the foregoing capabilities of the storage platform and to GUID. The Properties of a Type may be defined using any of access the data described in the schemas. The storage plat the primitive types or (with some restrictions noted below) form API 322 may be used by application programs in com any of the constructed types. For example a Location Type bination with other APIs, such as the OLE DB API 324 and might be defined that had Properties Coordinate and Address the Microsoft Windows Win32 API 326. 45 where the Address Property is of Type USPostalAddress as The storage platform 300 of the present invention may described above. Properties may also be required or optional. provide a variety of services 328 to application programs, Relationships can be declared and represent a mapping including a synchronization service 330 that facilitates the between the sets of instances of two types. For example there sharing of data among users or systems. For example, the may be a Relationship declared between the Person Type and synchronization service 330 may enable interoperability with 50 the Location Type called Lives.At which defines which people other data stores 340 having the same format as data store live at which locations. The Relationship has a name, two 302, as well as access to data stores 342 having other formats. endpoints, namely a source endpoint and a target endpoint. The storage platform 300 also provides file system capabili Relationships may also have an ordered set of properties. ties that allow interoperability of the data store 302 with Both the Source and Target endpoints have a Name and a existing file systems, such as the Windows NTFS files system 55 Type. For example the Lives.At Relationship has a Source 318. In at least some embodiments, the storage platform 320 called Occupant of Type Person and a Target called Dwelling may also provide application programs with additional capa of Type Location and in addition has properties StartDate and bilities for enabling data to be acted upon and for enabling EndDate indicating the period of time for which the occupant interaction with other systems. These capabilities may be lived at the dwelling. Note that a Person may live at multiple embodied in the form of additional services 328, such as an 60 dwellings over time and a dwelling may have multiple occu Info Agent service 334 and a notification service 332, as well pants so the most likely place to put the StartDate and End as in the form of other utilities 336. Date information is on the relationship itself. In at least some embodiments, the storage platform is Relationships define a mapping between instances that is embodied in, or forms an integral part of the hardware/ constrained by the types given as the endpoint types. For Software interface system of a computer system. For example, 65 example the LiveSAt relationship cannot be a relationship in and without limitation, the storage platform of the present which an Automobile is the Occupant because an Automobile invention may be embodied in, or form an integral part of an is not a Person. US 7,743,019 B2 17 18 The data model does allow the definition of a subtype derived from one or more simple data types and/or from other Supertype relationship between types. The Subtype-Supertype complex types. The complex types for the properties of an relationship also known as the BaseType relationship is Item also constitute “nested elements' since the details of the defined in such a way that if Type A is a BaseType for Type B complex type are nested into the immediate Item to define its it must be the case that every instance of B is also an instance 5 properties, and the information pertaining to these complex of A. Another way of expressing this is that every instance that types is maintained with the Item that has these properties conforms to B must also conform to A. If, for example A has (within the Item's boundary, as discussed later herein). These a property Name of Type String while B has a property Age of concepts of typing are well known and readily appreciated by Type Int16, it follows that any instance of B must have both a those of skill in the art. Name and an Age. The type hierarchy may be envisaged as an 10 FIG. 5B is a block diagram illustrating the complex prop tree with a single supertype at the root. The branches from the erty types Postal Address and EAddress. The Postal Address root provide the first level subtypes, the branches at this level property type defines that an Item of property type PostalAd provide the second level subtypes and so on to the leaf-most dress can be expected to have Zero or one City values, Zero or subtypes which themselves do not have any subtypes. The one CountryCode values, Zero or one MailStop values, and tree is not constrained to be of a uniform depth but cannot 15 any number (Zero to many) of PostalAddressTypes, and so on contain any cycles. A given Type may have Zero or many and so forth. In this way, the shape of the data for a particular Subtypes and Zero or one Super type. A given instance may property in an Item is hereby defined. The EAddress property conform to at most one type together with that type's Super type is similarly defined as shown. Although optionally used types. To put it another way, for a given instance at any level herein this Application, another way to represent the complex in the tree the instance may conform to at most one subtype at types in the Location Item is to draw the Item with the indi that level. A type is said to be Abstract if instances of the type vidual properties of each complex type listed therein. FIG.5C must also be an instance of a Subtype of the type. is a block diagram illustrating the Location Item wherein its 1. Items complex types are further described. However, it should be An Item is a unit of storable information that, unlike a understood that this alternative representation of the Location simple file, is an object having a basic set of properties that are 25 Item in this FIG. 5C is for the exact same Item illustrated in commonly Supported across all objects exposed to an end FIG. 5A. The storage platform of the present invention also user or application program by the storage platform. Items allows subtyping whereby one property type can be a Subtype also have properties and relationships that are commonly of another (where the one property type inherits the properties Supported across all Item types including features that allow of another, parent property type). new properties and relationships to be introduced, as dis 30 Similar to but distinct from properties and their property cussed below. types, Items inherently represent their own Item Types that Items are the objects for common operations such as copy, can also be the subject of subtyping. In other words, the delete, move, open, print, backup, restore, replicate, and so storage platform in several embodiments of the present inven forth. Items are the units that can be stored and retrieved, and tion allows an Item to be a subtype of another Item (whereby all forms of storable information manipulated by the storage 35 the one Item inherits the properties of the other, parent Item). platform exist as Items, properties of Items, or Relationships Moreover, for various embodiments of the present invention, between Items, each of which is discussed in greater detail every Item is a subtype of the “Item' Item type which is the herein below. first and foundational Item type found in the Base Schema. Items are intended to represent real-world and readily (The Base Schema will also be discussed in detail later understandable units of data like Contacts, People, Services, 40 herein.) FIG. 6A illustrates an Item, the Location Item in this Locations, Documents (of all various sorts), and so on. FIG. Instance, as being a subtype of the ItemItem type found in the 5A is a block diagram illustrating the structure of an Item. The Base Schema. In this drawing, the arrow indicates that the unqualified name of the Item is “Location'. The qualified Location Item (like all other Items) is a subtype of the Item name of the Item is “Core. Location' which indicates that this Item type. The Item Item type, as the foundational Item from Item structure is defined as a specific type of Item in the Core 45 which all other Items are derived, has a number of important Schema. (The Core Schema is discussed in more detail later properties such as ItemId and various timestamps, and herein.) thereby defines the standard properties of all Items in an The Location Item has a plurality of properties including operating system. In the present figure, these properties of the EAddresses, MetropolitanRegion, Neighborhood, and Posta Item Item type are inherited by Location and thereby become 1Addresses. The specific type of property for each is indicated 50 properties of Location. immediately following the property name and is separated Another way to represent the properties in the Location from the property name by a colon (":"). To the right of the Item inherited from the Item Item type is to draw Location type name, the number of values permitted for that property with the individual properties of each property type from the type is indicated between brackets (“I’) wherein an asterisk parent Item listed therein. FIG. 6B is a block diagram illus (*) to the right of the colon (":") indicates an unspecified 55 trating the Location Item wherein its inherited types and/or unlimited number (“many”). A “1” to the right of the described in addition to its immediate properties. It should be colon indicates that there can be at most one value. A Zero noted and understood that this Item is the same Item illus (“0”) to the left of the colon indicates that the property is trated in FIG. 5A, although in the present figure Location is optional (there may be no value at all). A “1” to the left of the illustrated with all of its properties, both immediate—shown colon indicates that there must be at least one value (the 60 in both this figure and FIG. 5A and inherited—shown in property is required). Neighborhood and Metropolitan Re this figure but not FIG. 5A (whereas in FIG. 5A these prop gion are both of type “nvarchar” (or equivalent) which is a erties are referenced by showing with an arrow that the Loca predefined data type or “simple type' (and denoted herein by tion Item is a subtype of the Item Item type). the lack of capitalization). EAddresses and PostalAddresses, Items are stand-alone objects; thus, if you delete an Item, however, are properties of defined types or “complex types” 65 all of the Items immediate and inherited properties are also (as denoted herein by capitalization) of types EAddress and deleted. Similarly, when retrieving an Item, what is received Postal Address respectively. A complex type is type that is is the Item and all of its immediate and inherited properties US 7,743,019 B2 19 20 (including the information pertaining to its complex property This type of reference cannot be used in a set operation. types). Certain embodiments of the present invention may The reference must generally be resolved through a path enable one to request a Subset of properties when retrieving a resolution process. The Resolve method of the storage plat specific Item; however, the default for many such embodi form API 322 provides this functionality. ments is to provide the Item with all of its immediate and 5 The reference forms discussed above are represented inherited properties when retrieved. Moreover, the properties through the reference type hierarchy illustrated in FIG. 11. of Items can also be extended by adding new properties to the Additional reference types that inherit from these types can existing properties of that Items type. These “extensions” are be defined in the schemas. They can be used in a relationship thereafter bona fide properties of the Item and subtypes of that declaration as type of the target field. Item type may automatically include the extension properties. 10 3. Item Folders and Categories The “boundary of the Item is represented by its properties As discussed more fully below, groups of Items can are (including complex property types, extensions, and so forth). organized into special Items called Item Folders (which are An Item's boundary also represents the limit of an operation not to be confused with file folders). Unlike in most file performed on an Item Such as copy, delete, move, create, and systems, however, an Item can belong to more than one Item so on. For example, in several embodiments of the present 15 Folder, such that when an Item is accessed in one Item Folder invention, when an Item is copied, everything within that and revised, this revised Item can then be accessed directly Item's boundary is also copied. For each Item, the boundary from another Item folder. In essence, although access to an encompasses the following: Item may occur from different Item Folders, what is actually The Item Type of the Item and, if the Item is a subtype of being accessed is in fact the very same Item. However, an another Item (as is the case in several embodiments of 20 Item Folder does not necessarily own all of its member Items, the present invention where all Items are derived from a or may simply co-own. Items in conjunction with other fold single Item and Item Type in the Base Schema), any ers, such that the deletion of an Item Folder does not neces applicable Subtype information (that is, information per sarily result in the deletion of the Item. Nevertheless, in sev taining to the parent Item Type). If the original Item eral embodiments of the present invention, an Item must being copied is a Subtype of another Item, the copy may 25 belong to at least one Item Folder so that if the sole Item also be a subtype of that same Item. Folder for a particular Item is deleted then, for some embodi The Items complex-type properties and extensions, if any. ments, the Item is automatically deleted or, in alternative If the original Item has properties of complex types embodiments, the Item automatically becomes a member of a (native or extended), the copy may also have the same default Item Folder (e.g., a “Trash Can' Item Folder concep complex types. 30 tually similar to similarly-named folders used in various file The Item's records on “ownership relationships, that is, and-folder-based systems). the Item's own list of what other Items (the “Target As also discussed more fully below, Items may also belong Items) are owned by the present Item (the “Owning to Categories based on common described characteristic Such Item'). This is particularly relevant in regard to Item as (a) an Item Type (or Types), (b) a specific immediate or Folders, discussed more fully below, and the rule stated 35 inherited property (or properties), or (c) a specific value (or below that all Items must belong to at least one Item values) corresponding to an Item property. For example, a Folder. Moreover, in regard to embedded items—dis Item comprising specific properties for personal contact cussed more fully below—an embedded item is consid information might automatically belong to a Contact Cat ered to be part of the Item in which it is embedded for egory, and any Item having contact information properties operations such as copy, delete, and the like. 40 would likewise automatically belong to this Category. Like 2. Item Identification wise, any Item having a location property with a value of Items are uniquely identified within the global items space “New York City’ might automatically belong to a NewY with an ItemD. The Base. Item type defines a field ItemID of orkCity Category. type GUID that stores the identity for the Item. An Item must Categories are conceptually different form Item Folders in have exactly one identity in the data store 302. 45 that, whereas Item Folders may comprise Items that are not An item reference is a data structure that contains informa interrelated (i.e., without a common described characteristic), tion to locate and identify an Item. In the data model, an each Itemina Category has a common type, property, or value abstract type is defined named ItemReference from which all (a “commonality”) that is described for that Category, and it item reference types derive. The ItemReference type defines is this commonality that forms the basis for its relationship to a virtual method named Resolve. The Resolve method 50 and among the other Items in the Category. Moreover, resolves the ItemReference and returns an Item. This method whereas an Item's membership in a particular Folder is not is overridden by the concrete subtypes of ItemReference, compulsory based on any particular aspect of that Item, for which implement a function that retrieves an Item given a certain embodiments all Items having a commonality cat reference. The Resolve method is invoked as part of the egorically related to a Category might automatically become storage platform API 322. 55 a member of the Category at the hardware/software interface ItemIDReference is a subtype of ItemReference. It defines system level. Conceptually, Categories can also be thought of a Locator and an ItemID field. The Locator field names (i.e. as virtual Item Folders whose membership is based on the identifies) an item domain. It is processed by a locator reso results of a specific query (such as in the context of a data lution method that can resolve the value of the Locator to an base), and Items that meet the conditions of this query (de item domain. The ItemID field is of type ItemID 60 fined by the commonalities of the Category) would thus com ItemPathReference is a specialization of ItemReference prise the Category's membership. that defines a Locator and a Path field. The Locator field FIG. 4 illustrates the structural relationship between Items, identifies an item domain. It is processed by a locator resolu Item Folders, and Categories. A plurality of Items 402, 404, tion method that can resolve the value of the Locator to an 406, 408, 410, 412, 414, 416, 418, and 420 are members of item domain. The Path field contains a (relative) path in the 65 various Item Folders 422,424, 426,428, and 430. Some Items storage platform namespace rooted at the item domain pro may belong to more than one Item Folder, e.g., Item 402 vided by the Locator. belong to Item Folders 422 and 424. Some Items, e.g., Item US 7,743,019 B2 21 22 402,404, 406, 408, 410, and 412 are also members of one or Schema defines a set of core Item types that, directly (by Item more Categories 432, 434, and 436, while other times, e.g., type) or indirectly (by Item subtype), characterize all Items Items 414, 416, 418, and 420, may belong to no Categories into one or more Core Schema Item types which the Item (although this is largely unlikely in certain embodiments based hardware/software interface system understands and where the possession of any property automatically implies can directly process in a predetermined and predictable way. membership in a Category, and thus an Item would have to be The predefined Item types reflect the most common Items in completely featureless in order not to be a member of any the Item-based hardware/software interface system and thus a category in Such an embodiment). In contrast to the hierar level of efficiency is gained by the Item-based hardware/ chical structure of folders, both Categories and Item Folders Software interface system understanding these predefined have structures more akin to directed graphs as shown. In any 10 Item types that comprise the Core Schema. event, the Items, Item Folders, and Categories are all Items In certain embodiments, the Core Schema is not extend (albeit of different Item Types). able—that is, no additional Item types can be subtyped In contrast to files, folders, and directories, the Items, Item directly from the Item type in the Base Schema except for the Folders, and Categories of the present invention are not char specific predefined derived Item types that are part of the Core acteristically “physical' in nature because they do not have 15 Schema. By preventing extensions to the Core Schema (that conceptual equivalents of physical containers, and therefore is, by preventing the addition of new Items to the Core Items may exist in more than one such location. The ability Schema), the storage platform mandates the use of the Core for Items to exist in more than one Item Folder location as Schema Item types since every Subsequent Item type is nec well as being organized into Categories provides an enhanced essarily a subtype of a Core Schema Item type. This structure and enriched degree of data manipulation and storage struc enables a reasonable degree of flexibility in defining addi ture capabilities at the hardware/software interface level, tional Item types while also preserving the benefits of having beyond that currently available in the art. a predefined set of core Item types. 4. Schemas For various embodiments of the present invention, and in a) Base Schema reference to FIG. 8A, the specific Item types supported by the To provide a universal foundation for the creation and use 25 Core Schema may include one or more of the following: of Items, various embodiments of the storage platform of the Categories: Items of this Item Type (and subtypes derived present invention comprise a Base Schema that establishes a therefrom) represent valid Categories in the Item-based conceptual framework for creating and organizing Items and hardware/software interface system. properties. The Base Schema defines certain special types of Commodities: Items that are identifiable things of value. Items and properties, and the features of these special foun 30 dational types from which subtypes can be further derived. Devices: Items having a logical structure that Supports The use of this Base Schema allows a programmer to concep information processing capabilities. tually distinguish Items (and their respective types) from Documents: Items with content that is not interpreted by properties (and their respective types). Moreover, the Base the Item-based hardware/software interface system but Schema sets forth the foundational set of properties that all 35 is instead interpreted by an application program corre Items may possess as all Items (and their corresponding Item sponding to the document type. Types) are derived from this foundational Item in the Base Events: Items that record certain occurrences in the envi Schema (and its corresponding Item Type). rOnment. As illustrated in FIG. 7, and in regard to several embodi Locations: Items representing physical locations (e.g., ments of the present invention, the Base Schema defines three 40 geographical locations). top-level types: Item, Extension, and PropertyBase. As Messages: Items of communication between two or more shown, the Item type is defined by the properties of this principals (defined below). foundational “Item' Item type. In contrast, the top level prop Principals: Items having at least one definitively provable erty type “Property Base' has no predefined properties and is identity aside from an ItemId (e.g., the identification of merely the anchor from which all other property types are 45 a person, organization, group, household, authority, Ser derived and through which all derived property types are vice, etc.). interrelated (being commonly derived from the single prop Statements: Items having special information regarding erty type). The Extension type properties define which Item the environment including, without limitation, policies, the extension extends as well as identification to distinguish Subscriptions, credentials, and so forth. one extension from another as an Item may have multiple 50 Likewise, and in reference to FIG.8B, the specific property extensions. types Supported by the Core Schema may include one or more ItemFolder is a subtype of the Item Item type that, in of the following: addition to the properties inherited from Item, features a Certificates (derived from the foundational PropertyBase Relationship for establishing links to its members (if any), type in the Base Schema) whereas both IdentityKey and Property are subtypes of Prop 55 ertyBase. CategoryRef in turn, is a subtype of IdentityKey. Principal Identity Keys (derived from the IdentityKey type b) Core Schema in the Base Schema) Various embodiments of the storage platform of the present Postal Address (derived from the Property type in the Base invention further comprise a Core Schema that provides a Schema) conceptual framework for top-level Items type structures. 60 Rich Text (derived from the Property type in the Base FIG. 8A is a block diagram illustrating Items in the Core Schema) Schema, and FIG. 8B is a block diagram illustrating the EAddress (derived from the Property type in the Base property types in the Core Schema. The distinction made Schema) between files with different extensions (*.com, *.exe, *.bat, IdentitySecurityPackage (derived from the Relationship *.sys, etc.) and other such criteria in file-and-folder-based 65 type in the Base Schema) systems is analogous to the function of the Core Schema. In RoleOccupancy (derived from the Relationship type in the the Item-based hardware/software interface system, the Core Base Schema) US 7,743,019 B2 23 24 BasicPresence (derived from the Relationship type in the relates. The storage platform API 322 provides mechanisms Base Schema) These Items and Properties are further for exposing relationships associated with an Item. described by their respective properties set forth in FIG. Here is an example of a relationship declaration: 8A and FIG. 8B. 5. Relationships Relationships are binary relationships where one Item is designated as source and the other Item as target. The Source Item and the target Item are related by the relationship. The ship. That is, when the source Item is deleted, the relationship 10 illustrates the manner in which relationships are classified. The Containment relationship types are further classified into Holding and Embedding relationships. When all holding This is an example of a Reference relationship. The rela relationships to an Item are removed, the Item is deleted. A tionship can not be created if the person Item that is refer holding relationship controls the life-time of the target enced by the source reference does not exist. Also, if the through a reference counting mechanism. The embedding person Item is deleted, the relationship instances between the relationships enable modeling of compound Items and can be person and organization are deleted. However, if the Organi thought of as exclusive holding relationships. An Item can be zation Item is deleted, the relationship is not deleted and it is a target of one or more holding relationships; but an Item can dangling. be target of exactly one embedding relationship. An Item that 25 b) Holding Relationship is a target of an embedding relationship can not be a target of Holding relationships are used to model reference count any other holding or embedding relationships. based life-time management of the target Items. Reference relationships do not control the lifetime of the target Item. They may be dangling—the target Item may not An Item can be a source endpoint for Zero or more rela exist. Reference relationships can be used to model refer 30 tionships to Items. An Item that is not an embedded Item can ences to Items anywhere in the global Item name space (i.e. be a target of in one or more holding relationships. including remote data stores). The target endpoint reference type must be ItemIDRefer Fetching an Item does not automatically fetch its relation ence and it must reference an Item in the same store as the ships. Applications must explicitly request the relationships relationship instance. of an Item. In addition, modifying a relationship does not 35 Holding relationships enforce lifetime management of the modify the Source or the target Item; similarly, adding a target endpoint. The creation of a holding relationship relationship does not affect the source/target Item. instance and the Item that it is targeting is an atomic opera a) Relationship Declaration tion. Additional holding relationship instances can be created The explicit relationship types are defined with the follow that are targeting the same Item. When the last holding rela ing elements: 40 tionship instance with a given Item as target endpoint is A relationship name is specified in the Name attribute. deleted the target Item is also deleted. Relationship type, one of the following: Holding, Embed The types of the endpoint Items specified in the relation ding, Reference. This is specified in the Type attribute. ship declaration will generally be enforced when an instance Source and target endpoints. Each endpoint specifies a of the relationship is created. The types of the endpoint Items name and the type of the referenced Item. 45 The source endpoint field is generally of type ItemID (not can not be changed after the relationship is established. declared) and it must reference an Item in the same data Holding relationships play a key role in forming the Item store as the relationship instance. namespace. They contain the “Name” property that defines For Holding and Embedding relationships, the target end the name of the target Item relative to the source Item. This point field must be of type ItemIDReference and it must 50 relative name is unique for all the holding relationships reference an Item in the same store as the relationship sourced from a given Item. The ordered list of this relative instance. For Reference relationships the target endpoint names starting from the root Item to a given Item forms the can be of any ItemReference type and can reference full name to the Item. Items in other storage platform data stores. The holding relationships form a directed acyclic graph Optionally one or more fields of a scalar or PropertyBase 55 (DAG). When a holding relationship is created the system type can be declared. These fields may contain data ensures that a cycle is not created, thus ensuring that the Item associated with the relationship. namespace forms a DAG. Relationship instances are stored in a global relationships While the holding relationship controls the life time of the table. target Item, it does not control the operational consistency of Every relationship instance is uniquely identified by the 60 the target endpoint Item. The target Item is operationally combination (source ItemID, relationship ID). The rela independent from the Item that owns it through a holding tionship ID is unique within a given source ItemID for relationship. Copy, Move, Backup and other operations on an all relationships sourced in a given Item regardless of Item that is a source of a holding relationship do not affect the their type. Item that is a target of the same relationship—for example The source Item is the owner of the relationship. While an 65 that is, backing up a Folder Item does not automatically Item designated as owner controls the life time of the rela backup all the Items in the folder (targets of the FolderMem tionship, the relationship itself is separate from the Items it ber relationship). US 7,743,019 B2 25 26 The following is an example of a holding relationship: -continued platformTypes.nvarchar(256)' > Any reference type is allowed in the target endpoint. The Items that participate in a reference relationship can be of any Item type. The FolderMembers relationship enables the concept of a 10 Reference relationships are used to model most non-life Folder as a generic collection of Items. time management relationships between Items. Since the c) Embedding Relationships existence of the target is not enforced, the reference relation Embedding relationships model the concept of exclusive ship is convenient to model loosely-coupled relationships. control of the lifetime of the target Item. They enable the The reference relationship can be used to target Items in other concept of compound Items. 15 data stores including stores on other computers. The creation of an embedding relationship instance and the e) Rules and Constraints Item that it is targeting is an atomic operation. An Item can be The following additional rules and constraints apply for a source of Zero or more embedding relationship. However, relationships: an Item can be a target of one and only one embedding An Item must be a target of (exactly one embedding rela relationship. An Item that is a target of an embedding rela tionship) or (one or more holding relationships). One tionship can not be a target of a holding relationship. exception is the root Item. An Item can be a target of Zero The target endpoint reference type must be ItemIDRefer or more reference relationships ence and it must reference an Item in the same data store as the An Item that is a target of embedding relationship can not relationship instance. be source of holding relationships. It can be a source of The types of the endpoint Items specified in the relation 25 reference relationships. ship declaration will generally be enforced when an instance An Item can not be a source of holding relationship if it is of the relationship is created. The types of the endpoint Items promoted from file. It can be a source of embedding can not be changed after the relationship is established. relationships and reference relationships. Embedding relationships control the operational consis An Item can that is promoted from a file can not be a target tency of the target endpoint. For example the operation of 30 of an embedding relationship. serializing of an Item may include serialization of all the f) Ordering of Relationships embedding relationships that source from that Item as well as In at least one embodiment, the storage platform of the all of their targets; copying an Item also copies all its embed present invention Supports ordering of relationships. The ded Items. ordering is achieved through a property named “Order in the The following is an example declaration: 35 base relationship definition. There is no uniqueness constraint on the Order field. The order of the relationships with the same “order” property value is not guaranteed, however it is guaranteed that they may be ordered after relationships with “order field value. Applications can get the relationships in the default order shipID, Order). All relationship instances sourced from a given Item are ordered as a single collection regardless of the 45 type of the relationships in the collection. This however guar antees that all relationships of a given type (e.g., FolderMem bers) are an ordered subset of the relationship collection for a d) Reference Relationships given Item. The reference relationship does not control life time of the The data store API 312 for manipulating relationships Item it references. Even more, the reference relationships do 50 implement a set of operations that Support ordering of rela not guarantee the existence of the target, nor do they guaran tionships. The following terms are introduced to help explain tee the type of the target as specified in the relationship the operations: declaration. This means that the reference relationships can RelFirst is the first relationship in the ordered collection be dangling. Also, the reference relationship can reference with order value OrdEirst; Items in other data stores. Reference relationships can be 55 RelLast is the last relationship in the ordered collection thought of as a concept similar to links in web pages. with order value OrdLast; An example of reference relationship declaration is the RelX is a given relationship in the collection with order following: value OrdX; RelPrev is a closest relationship in the collection to RelX 60 with order value OrdPrev Smaller then OrdX; and RelNext is a closest relationship in the collection to RelX InsertBeforeFirst(SourceltemID, Relationship) inserts the 1006 and Item 1006 has a Relationship 1026 back to Category Finally, because Categories and Item Folders are them selves Items, and Items may Relationship to each other, Cat 55 egories may Relationship to Item Folders and vice versa, and The ItemID field contains the ItemID of the item that the Categories, Item Folders, and Items can Relationship to other extension is associated with. An Item with this ItemID must Categories, Item Folders, and Item respectively in certain exist. The extension can not be created if the item with the alternative embodiments. However, in various embodiments, given ItemID does not exist. When the Item is deleted all the Item Folder structures and/or Category structures are prohib 60 extensions with the same ItemID are deleted. The tuple ited, at the hardware/software interface system level, from (ItemID,ExtensionID) uniquely identifies an extension containing cycles. Where Item Folder and Category struc instance. tures are akin to directed graphs, the embodiments that pro The structure of an extension type is similar to that of an hibit cycles areakin to directed acyclic graphs (DAGs) which, item type: by mathematical definition in the art of graph theory, are 65 Extension types have fields: directed graphs wherein no path starts and ends at the same Fields can be of primitive or nested element types; and Vertex. Extension types can be sub-typed. US 7,743,019 B2 31 32 The following restrictions apply for extension types platform. The application developer would define a CRM Extensions can not be sources and targets of relationships: extension that would contain the additional data structure that Extension type instances can not exist independently from the application can manipulate. an item; and Extension types can not be used as field types in the storage platform type definitions There are no constraints on the types of extensions that can extension type creates a new extension type. The structure or not automatically retrieved. Given a Contact item, its related the same item id. In the above example, the Contact item

Item vs. Item Extension vs NestedElement

Item Item Extension NestedElement

ItemID Has its own item id Shares the item id Does not have its of the item own item id. Nested element is part of the item Storage Item hierarchy is Item extension Stored with item stored in its own hierarchy is stored tables in its own tables Query/Search Can query item Can query item Can generally be tables extension tables queried only within the containing item context US 7,743,019 B2 33 34

-continued

Item vs. Item Extension vs NestedElement

Item Item Extension NestedElement Query/Search Can Search across Can Search across Can generally only Scope all instances of an all instances of an search within nested item type item extension type element type instances of a singe (containing) item Relationship Can have Rela- No Relation- No Relationships to Semantics tionships to ships to item nested elements items extensions Association Can be related to Can generally only Related to item via to items other items via be related via fields. Nested holding, embedded extensions. The elements are part of and soft extension semantics the item Relationships is similar to embedded item Semantics

b) Extending NestedElement Types The following table Summarizes and compares Item Exten Nested Element types are not extended with the same sions and NestedElement extensions. mechanism as the Item types. Extensions of nested elements are stored and accessed with the same mechanisms as fields of nested element types. 25 The data model defines a root for nested element types Item extensions vs Nested Element extensions named Element: Item Extension NestedElement Extension

Storage Item extension Stored like 30 hierarchy is nested elements 35 Query/Search Can Search across all Can generally only search Scope instances of an item within nested element type extension type instances of a singe (containing) item The Nested Element type inherits from this type. The Nest- Programmability Need special extension NestedElement extensions edElement element type additionally defines a field that is a APIsquerying and Onspecial valuedare like field any otherof nested multi multi-set of Elements. 40 extension tables element; normal nested element type APIs are used Behavior Can associate behavior No behavior permitted (?) Relationship No Relationships No Relationships to

Object Name Decoration Description Example Master Item Master Item Provides a System.Storage. Search View Summary of items Master! Item in the current item domain. Typed Item ItemType Provides all AcmeCorp. Doc. search view property data OfficeDoc) from item and any parent type(s). Master MasterExtension Provides a System.Storage. Extension Summary of all MasterExtension Search View extensions in the current item domain. Typed Extension!extensionType Provides al AcmeCorp. Doc. extension property data for Extension! StickyNote search view extension. Master Master! Relationship Provides a System.Storage. Relationship Summary of all Master! Relationship View relationships in the current item domain. Relationship Relationship relationship Provides all data AcmeCorp. Doc. view Name associated with a Relationship AuthorsRrom given relationship Document View ViewviewName Provides the AcmeCorp. Doc. columns, types View! DocumentTitles based on the Schema view definition.

7. Column Naming Logical “key' column (s) of view result such as ItemId, When mapping any object model into a store, the possibil ElementId, RelationshipId, ... ity of naming collisions occur due to additional information 60 Metadata information on type of result such as Typeld. stored along with an application object. In order to avoid naming collisions, all non-type specific columns (columns Change tracking columns such as CreateVersion, Updat which do not map directly to a named Property in a type eVersion, ... declaration) is be prefixed with an underscore ( ) character. In as Type specific column(s) (Properties of the declared type) the present embodiment, underscore ( ) characters are disal Type specific views (family views) also contain an object lowed as the beginning character of any identifier property. column which returns the object US 7,743,019 B2 39 40 Members of each type family are searchable using a series b) Item Extensions of Item views, with there being one view per Item type in the All Item Extensions in a WinFS Store are also accessible data store. FIG. 28 is a diagram illustrating the concept of an using search views. Item search view. (1) Master Extension Search View a) Item Each Item search view contains a row for each instance of Each instance of a data store defines a special Extension an Item of the specific type or its subtypes. For example, the view called the Master Extension View. This view provides view for Document could return instances of Document, Summary information on each Extension in the data store. LegalDocument and ReviewDocument. Given this example, 10 The view has a column per Extension property, a column the Item views can be conceptualized as shown in FIG. 29. which describes the type of the Extension and several col umns which are used to provide change tracking and synchro nization information. The master extension view is identified (1) Master Item SearchView in a data store using the name "System.Storage. Master 15 Extension'. Each instance of a storage platform data store defines a special Item view called the Master Item View. This view provides Summary information on each Item in the data store. The view provides one column per Item type property, a Column Type Description column which described the type of the Item and several ItemId ItemId The storage platform identity of the Item with which columns which are used to provide change tracking and Syn this extension is associated chronization information. The master item view is identified ExtensionId ExtensionId Id of this extension instance in a data store using the name "System.Storage. Master (GUID) TypeId TypeId The TypeId of the Extension - Item'. 25 identifies the exact type of the extension and can be used to retrieve information on the extension using the Column Type Description Metadata catalog.

(2) Typed Item Search Views 45 Column Type Description Each Item type also has a search view. While similar to the ItemId ItemId The storage platform identity root Item view, this view also provides access to the Item of the Item with which this extension is associated object via the “ Item’ column. Each typed item search view ExtensionId ExtensionId Id of this extension instance is identified in a data store using the name schemaName. 50 (GUID) itemTypeName. For example AcmeCorp. Doc. Office specific- property ItemId Named property of 50 Item type is available from the Master Item View “Sys Source endpoint tem.Storage. Master. Item' and typed Item search view identity AcmeCorp. Document. Document. (alias for ItemId) ItemReference or Named property of derived class target endpoint (1) Change Tracking in “Master Search Views identity 55 (alias and cast for TargetItemReference) Change tracking information in the master search views extensions and relationships) in one place. The item and provides access to the relationship object (the relationship (1) Item Tombstones Item tombstones are retrieved from the system via the view 15 System.Storage.Tombstone. Item.

These properties contain the following information: 2O Column Type Description ItemId ItemID Identity of the Item Column Description TypeID TypeId Type of the Item

(2) Change Tracking in “Typed Search Views (2) Extension Tombstones 45 In addition to providing the same information as the global Extension tombstones are retrieved from the system using search view, each typed search view provides additional the view System. Storage.Tombstone Extension. Exten information recording the sync state of each element in the sion change tracking information is similar to that provided Sync topology. for Items with the addition of the ExtensionID property.

Column Type Description

Column Type Description Returns an Item object given an ItemId ItemID ItemID Identity of the Item which owns the Extension Item GetItem (ItemId ItemId) ExtensionId ExtensionId Extension Id of the Extension TypeID TypeId Type of the extension b) Function System.Storage.GetExtension ChangeTrackingInfo CLR instance of Change tracking infor type mation for this extension 10 ChangeTrackingInfo Is Deleted BIT This is a flag that is 0 for live items, and 1 for if Returns an extension object given an ItemId and ExtensionId tombstoned extensions. DeletionWallclock UTCDATETIME The UTC wall clock date Extension GetExtension (ItemId ItemId, ExtensionId ExtensionId) time according to the 15 partner which deleted the extension. It is NULL c) Function System.Storage.GetRelationship if the extension is live.

(3) Relationships Tombstone if Returns an relationship object given an ItemId and RelationshipId Relationship GetRelationship (ItemId ItemId, RelationshipId Relationship tombstones are retrieved from the system via RelationshipId) the view System. Storage.Tombstone Relationship. Rela tionships tombstone information is similar to that provided 12. Metadata for Extensions. However, additional information is provided 25 There are two types of metadata represented in the Store: on the target ItemRefof the relationship instance. In addition, instance metadata (the type of an Item, etc), and type meta the relationship object is also selected. data. a) Schema Metadata 30 Schema metadata is stored in the data store as instances of Column Description Item types from the Meta schema. ItemID ItemID Identity of the Item b) Instance Metadata which owned the rela Instance metadata is used by an application to query for the tionship (identity of type of an Item and finds the extensions associated with an relationship source 35 Item. Given the Itemld for an Item, an application can query endpoint) RelationshipId RelationshipId RelationshipId the global item view to return the type of the Item and use this of the relationship value to query the Meta. Type view to return information on TypeID TypeId Type of the the declared type of the Item. For example, relationship ChangeTrackingInfo CLR instance of Change tracking type information for 40 ChangeTrackingInfo his relationship if Return metadata Item object for given Item instance Is Deleted BIT This is a flag that is 0 or live items, and 1 for SELECT m. Item AS metadataInfoObj ombstoned extensions. FROM System. Storage). Item i INNER JOIN DeletionWallclock UTCDATETIME The UTC wall clock date Meta-Type m ON i. TypeId= m. ItemId ime according to the 45 partner which deleted WHERE i.ItemId= (a ItemId he relationship. t is NULL if the relationship is live. E. Security Relationship CLR instance of a This is the relationship In general, all securable objects arrange their access rights Relationship object for live rela 50 using the access mask format shown in the FIG. 26. In this ionship. It is NULL for ombstoned relationships. format, the low-order 16 bits are for object-specific access TargetItemReference ItemReference dentity of rights, the next 7 bits are for standard access rights, which arget endpoint apply to most types of objects, and the 4 high-order bits are used to specify generic access rights that each object type can 55 map to a set of standard and object-specific rights. The (4) Tombstone Cleanup ACCESS SYSTEM SECURITY bit corresponds to the right to access the object’s SACL. In order to prevent unbounded growth of tombstone infor In the access mask structure of FIG. 26, item specific rights mation, the data store provides a tombstone cleanup task. This are placed in the Object Specific Rights section (low order task determines when tombstone information may be dis 60 16-bits). Because in the present embodiment, the storage carded. The task computes a bound on the local create/update platform exposes two sets of APIs to administer security— version and then truncates the tombstone information by dis Win32 and the storage platform API, the file system object carding all earlier tombstone versions. specific rights must be considered in order to motivate the 11. Helper APIs and Functions design of the storage platform object specific rights. The Base mapping also provides a number of helper func 65 The security model for the storage platform of the present tions. These functions are Supplied to aid common operations invention is fully described in the related applications incor over the data model. porated by reference earlier herein. In this regard, FIG. 27 US 7,743,019 B2 47 48 (parts a, b, and c) depicts a new identically protected security H. Storage Platform API region being carved out of an existing security region, in The storage platform comprises an API that enables appli accordance with one embodiment of a security model. cation programs to access the features and capabilities of the F. Notifications and Change Tracking storage platform discussed above and to access items stored According to another aspect of the present invention, the in the data store. This section describes one embodiment of a storage platform provides a notifications capability that storage platform API of the storage platform of the present allows applications to track data changes. This feature is invention. Details regarding this functionality can be found in primarily intended for applications which maintain volatile the related applications incorporated by reference earlier state or execute business logic on data change events. Appli herein, with some of this information summarized below for cations register for notifications on items, item extensions and 10 convenience. item relationships. Notifications are delivered asynchro Referring to FIG. 18, a Containment Folder is an item nously after data changes have been committed. Applications which contains holding Relationships to other Items and is may filter notifications by item, extension and relationship the equivalent of the common concept of a file system folder. type as well as type of operation. Each Item is “contained within at least one containment According to one embodiment, the storage platform API 15 folder. 322 provides two kinds of interfaces for notifications. First, FIG. 19 illustrates the basic architecture of the storage applications register for simple data change events triggered platform API, in accordance with the present embodiment. by changes to items, item extensions and item relationships. The storage platform API uses SQLClient 1900 to talk to the Second, applications create “watcher' objects to monitor sets local data store 302 and may also use SQLClient 1900 to talk of items, item extensions and relationships between items. to remote data stores (e.g., data store 340). The local store 302 The state of a watcher object can be saved and re-created after may also talk to the remote data store 340 using either DQP a system failure or after a system has gone off-line for an (Distributed Query Processor) or through the storage plat extended period of time. A single notification may reflect form synchronization service ("Sync’) described below. The multiple updates. storage platform API 322 also acts as the bridge API for data Additional details regarding this functionality can be found 25 store notifications, passing application’s Subscriptions to the in the related applications incorporated by reference earlier notification engine 332 and routing notifications to the appli herein. cation (e.g., application 350a, 350b, or 350c), as also G. Traditional File System Interoperability described above. In one embodiment, the storage platform As mentioned above, the storage platform of the present API322 may also define a limited “provider architecture so invention is, in at least some embodiments, intended to be 30 that it can access data in Microsoft Exchange and AD. embodied as an integral part of the hardware/software inter FIG. 20 schematically represents the various components face system of a computer system. For example, the storage of the storage platform API. The storage platform API con platform of the present invention may be embodied as an sists of the following components: (1) data classes 2002, integral part of an operating system, Such as the Microsoft which represent the storage platform element and item types, Windows family of operating systems. In that capacity, the 35 (2) runtime framework 2004, which manages object persis storage platform API becomes a part of the operating system tence and provides support classes 2006; and (3) tools 2008, APIs through which application programs interact with the which are used to generate CLR classes from the storage operating system. Thus, the storage platform becomes the platform schemas. means through which application programs store information The hierarchy of classes resulting from a given schema on the operating system, and the Item based data model of the 40 directly reflects the hierarchy of types in that schema. As an storage platform therefore replaces the traditional files sys example, consider the Item types defined in the Contacts tem of such an operating system. For example, as embodied in schema as shown in FIG. 21A and FIG. 21B. the Microsoft Windows family of operating systems, the stor FIG.22 illustrates the runtime frameworkin operation. The age platform might replace the NTFS file system imple runtime framework operates as follows: mented in that operating system. Presently, application pro 45 1. An application 350a, 350b, or 350c binds to an item in grams access the services of the NTFS file system through the the storage platform. Win32 APIs exposed by the Windows family of operating 2. The framework 2004 creates an Item.Context object 2202 systems. corresponding to the bound item and returns it to the Recognizing, however, that completely replacing the application. NTFS file system with the storage platform of the present 50 3. The application submits a Find on this ItemContext to invention would require recoding of existing Win32-based get a collection of Items; the returned collection is con application programs and that Such recoding may be undesir ceptually an object graph 2204 (due to relationships). able, it would be beneficial for the storage platform of the 4. The application changes, deletes, and inserts data. present invention to provide some interoperability with exist 5. The application saves the changes by calling the ing file systems, such as NTFS. In one embodiment of the 55 Update() method. present invention, therefore, the storage platform enables FIG. 23 illustrates the execution of a “FindAll operation. application programs which rely on the Win32 programming FIG. 24 illustrates the process by which storage platform model to access the contents of both the data store of the API classes are generated from the storage platform Schema storage platform as well as the traditional NTFS file system. FIG. 25 illustrates the schema on which the File API is To this end, the storage platform uses a naming convention 60 based. The storage platform API includes a namespace for that is a superset of the Win32 naming conventions to facili dealing with file objects. This namespace is called Sys tate easy interoperability. Further, the storage platform Sup tem.Storage. Files. The data members of the classes in Sys ports accessing files and directories stored in a storage plat tem.Storage. Files directly reflect the information stored in the form volume through the Win32 API. storage platform store; this information is “promoted from Additional details regarding this functionality can be found 65 the file system objects or may be created natively using the in the related applications incorporated by reference earlier Win32 API. The System.Storage. Files namespace has two herein. classes: FileItem and DirectoryItem. The members of these US 7,743,019 B2 49 50 classes and methods thereof can be readily divined by looking coupled or tightly coupled configuration, and so this list at the schema diagram in FIG.25. FileItem and DirectoryItem should be considered illustrative and non-limiting. are read-only from the storage platform API. In order to This notion of a programming interface is known to those modify them, one has to use the Win32 API or classes in skilled in the art and is clear from the foregoing detailed System.IO. description of the invention. There are, however, other ways In regard to APIs, a programming interface (or more sim to implement a programming interface, and, unless expressly ply, interface) may be viewed as any mechanism, process, excluded, these too are intended to be encompassed by the protocol for enabling one or more segment(s) of code to claims set forth at the end of this specification. Such other communicate with or access the functionality provided by ways may appear to be more Sophisticated or complex than one or more other segment(s) of code. Alternatively, a pro 10 the simplistic view of FIGS. 30A and 30B, but they nonethe gramming interface may be viewed as one or more mecha less perform a similar function to accomplish the same overall nism(s), method(s), function call(s), module(s), object(s), etc. result. We will now briefly describe some illustrative alterna of a component of a system capable of communicative cou tive implementations of a programming interface. pling to one or more mechanism(s), method(s), function Factoring: A communication from one code segment to call(s), module(s), etc. of other component(s). The term “seg 15 another may be accomplished indirectly by breaking the com ment of code” in the preceding sentence is intended to include munication into multiple discrete communications. This is one or more instructions or lines of code, and includes, e.g., depicted schematically in FIGS. 31A and 31B. As shown, code modules, objects, Subroutines, functions, and so on, some interfaces can be described in terms of divisible sets of regardless of the terminology applied or whether the code functionality. Thus, the interface functionality of FIGS. 30A segments are separately compiled, or whether the code seg and 30B may be factored to achieve the same result, just as ments are provided as Source, intermediate, or object code, one may mathematically provide 24, or 2 times 2 time 3 times whether the code segments are utilized in a runtime system or 2. Accordingly, as illustrated in FIG. 31A, the function pro process, or whether they are located on the same or different vided by interface Interfacel may be subdivided to convert the machines or distributed across multiple machines, or whether communications of the interface into multiple interfaces the functionality represented by the segments of code are 25 Interface1A, Interface 1B, Interface 1C, etc. while achieving implemented wholly in software, wholly in hardware, or a the same result. As illustrated in FIG.31B, the function pro combination of hardware and software. vided by interface I1 may be subdivided into multiple inter Notionally, a programming interface may be viewed faces I1a, I1b, I1c, etc. while achieving the same result. generically, as shown in FIG. 30A or FIG. 30B. FIG. 30A Similarly, interface I2 of the second code segment which illustrates an interface Interfacel as a conduit through which 30 receives information from the first code segment may be first and second code segments communicate. FIG.30B illus factored into multiple interfaces I2a, I2b, I2C, etc. When trates an interface as comprising interface objects I1 and I2 factoring, the number of interfaces included with the 1st code (which may or may not be part of the first and second code segment need not match the number of interfaces included segments), which enable first and second code segments of a with the 2nd code segment. In either of the cases of FIGS.31A system to communicate via medium M. In the view of FIG. 35 and 31B, the functional spirit of interfaces Interface1 and I1 30B, one may consider interface objects I1 and I2 as separate remain the same as with FIGS. 30A and 30B, respectively. interfaces of the same system and one may also consider that The factoring of interfaces may also follow associative, com objects I1 and I2 plus medium M comprise the interface. mutative, and other mathematical properties such that the Although FIGS. 30A and 30B show bi-directional flow and factoring may be difficult to recognize. For instance, ordering interfaces on each side of the flow, certain implementations 40 of operations may be unimportant, and consequently, a func may only have information flow in one direction (or no infor tion carried out by an interface may be carried out well in mation flow as described below) or may only have an inter advance of reaching the interface, by another piece of code or face object on one side. By way of example, and not limita interface, or performed by a separate component of the sys tion, terms such as application programming interface (API), tem. Moreover, one of ordinary skill in the programming arts entry point, method, function, Subroutine, remote procedure 45 can appreciate that there are a variety of ways of making call, and (COM) interface, are different function calls that achieve the same result. encompassed within the definition of programming interface. Redefinition: In some cases, it may be possible to ignore, Aspects of Such a programming interface may include the add or redefine certain aspects (e.g., parameters) of a pro method whereby the first code segment transmits information gramming interface while still accomplishing the intended (where “information' is used in its broadest sense and 50 result. This is illustrated in FIGS. 32A and 32B. For example, includes data, commands, requests, etc.) to the second code assume interface Interfacel of FIG. 30A includes a function segment; the method whereby the second code segment call Square(input, precision, output), a call that includes three receives the information; and the structure, sequence, syntax, parameters, input, precision and output, and which is issued organization, Schema, timing and content of the information. from the 1st Code Segment to the 2nd Code Segment. If the In this regard, the underlying transport medium itself may be 55 middle parameter precision is of no concern in a given sce unimportant to the operation of the interface, whether the nario, as shown in FIG. 32A, it could just as well be ignored medium be wired or wireless, or a combination of both, as or even replaced with a meaningless (in this situation) param long as the information is transported in the manner defined eter. One may also add an additional parameter of no concern. by the interface. In certain situations, information may not be In either event, the functionality of square can be achieved, so passed in one or both directions in the conventional sense, as 60 long as output is returned after input is squared by the second the information transfer may be either via another mechanism code segment. Precision may very well be a meaningful (e.g. information placed in a buffer, file, etc. separate from parameter to some downstream or other portion of the com information flow between the code segments) or non-exis puting system; however, once it is recognized that precision is tent, as when one code segment simply accesses functionality not necessary for the narrow purpose of calculating the performed by a second code segment. Any or all of these 65 square, it may be replaced or ignored. For example, instead of aspects may be important in a given situation, e.g., depending passing a valid precision value, a meaningless value Such as a on whether the code segments are part of a system in a loosely birth date could be passed without adversely affecting the US 7,743,019 B2 51 52 result. Similarly, as shown in FIG. 32B, interface I1 is Java ByteCode, etc.) is provided to a Just-in-Time (JIT) com replaced by interface I1", redefined to ignore or add param piler or interpreter in an execution environment (such as that eters to the interface. Interface I2 may similarly be redefined provided by the Net framework, the Java runtime environ as interface I2', redefined to ignore unnecessary parameters, ment, or other similar runtime type environments). The JIT or parameters that may be processed elsewhere. The point compiler may be written so as to dynamically convert the here is that in Some cases a programming interface may communications from the 1st Code Segment to the 2nd Code include aspects, such as parameters, that are not needed for Segment, i.e., to conform them to a different interface as may Some purpose, and so they may be ignored or redefined, or be required by the 2nd Code Segment (either the original or a processed elsewhere for other purposes. different 2nd Code Segment). This is depicted in FIGS. 35A Inline Coding: It may also be feasible to merge some or all 10 and 35B. As can be seen in FIG.35A, this approach is similar of the functionality of two separate code modules such that to the Divorce scenario described above. It might be done, the “interface' between them changes form. For example, the e.g., where an installed base of applications are designed to functionality of FIGS. 30A and 30B may be converted to the communicate with an operating system in accordance with an functionality of FIGS. 33A and 33B, respectively. In FIG. Interface 1 protocol, but then the operating system is changed 33A, the previous 1st and 2nd Code Segments of FIG.30A are 15 to use a different interface. The JIT Compiler could be used to merged into a module containing both of them. In this case, conform the communications on the fly from the installed the code segments may still be communicating with each base applications to the new interface of the operating system. other but the interface may be adapted to a form which is more As depicted in FIG.35B, this approach of dynamically rewrit suitable to the single module. Thus, for example, formal Call ing the interface(s) may be applied to dynamically factor, or and Return statements may no longer be necessary, but simi otherwise alter the interface(s) as well. lar processing or response(s) pursuant to interface Interfacel It should also be noted that the above-described scenarios may still be in effect. Similarly, shown in FIG. 33B, part (or for achieving the same or similar result as an interface via all) of interface I2 from FIG. 30B may be written inline into alternative embodiments may also be combined in various interface I1 to form interface I1". As illustrated, interface I2 is ways, serially and/or in parallel, or with other intervening divided into I2a and I2b, and interface portion I2a has been 25 code. Thus, the alternative embodiments presented above are coded in-line with interface I1 to form interface I1". For a not mutually exclusive and may be mixed, matched and com concrete example, consider that the interface I1 from FIG. bined to produce the same or equivalent scenarios to the 30B performs a function call square (input, output), which is generic scenarios presented in FIGS. 30A and 30B. It is also received by interface I2, which after processing the value noted that, as with most programming constructs, there are passed with input (to square it) by the second code segment, 30 other similar ways of achieving the same or similar function passes back the squared result with output. In such a case, the ality of an interface which may not be described herein, but processing performed by the second code segment (squaring nonetheless are represented by the spirit and scope of the input) can be performed by the first code segment without a invention, i.e., it is noted that it is at least partly the function call to the interface. ality represented by, and the advantageous results enabled by, Divorce: A communication from one code segment to 35 an interface that underlie the value of an interface. another may be accomplished indirectly by breaking the com munication into multiple discrete communications. This is III. Synchronization API depicted schematically in FIGS. 34A and 34.B. As shown in Several approaches to synchronization are possible in an FIG.34A, one or more piece(s) of middleware (Divorce Inter Item-based hardware/software interface system. Section A face(s), since they divorcefunctionality and/or interfacefunc 40 discloses several embodiments of the present invention, while tions from the original interface) are provided to convert the Section B focuses on various embodiments of an API for communications on the first interface, Interface1, to conform synchronization. them to a different interface, in this case interfaces A. Synchronization Overview Interface2A, Interface2B and Interface2C. This might be done, e.g., where there is an installed base of applications 45 For several embodiments of the present invention, and in designed to communicate with, say, an operating system in regard to FIG. 3, the storage platform provides a synchroni accordance with an Interfacel protocol, but then the operating zation service 330 that (i) allows multiple instances of the system is changed to use a different interface, in this case storage platform (each with its own data store 302) to syn interfaces Interface2A, Interface2B and Interface2C. The chronize parts of their content according to a flexible set of point is that the original interface used by the 2nd Code 50 rules, and (ii) provides an infrastructure for third parties to Segment is changed Such that it is no longer compatible with synchronize the data store of the storage platform of the the interface used by the 1st Code Segment, and so an inter present invention with other data sources that implement mediary is used to make the old and new interfaces compat proprietary protocols. ible. Similarly, as shown in FIG. 34B, a third code segment Storage-platform-to-storage-platform synchronization can be introduced with divorce interface DI1 to receive the 55 occurs among a group of participating “replicas. For communications from interface I1 and with divorce interface example, with reference to FIG. 3, it may be desirable to DI2 to transmit the interface functionality to, for example, provide synchronization between the data store 302 of the interfaces I2a and I2b, redesigned to work with DI2, but to storage platform 300 with another remote data store 338 provide the same functional result. Similarly, DI1 and DI2 under the control of another instance of the storage platform, may work together to translate the functionality of interfaces 60 perhaps running on a different computer system. The total I1 and I2 of FIG. 30B to a new operating system, while membership of this group is not necessarily known to any providing the same or similar functional result. given replica at any given time. Rewriting. Yet another possible variant is to dynamically Different replicas can make the changes independently rewrite the code to replace the interface functionality with (i.e., concurrently). The process of synchronization is defined something else but which achieves the same overall result. 65 as making every replica aware of the changes made by other For example, there may be a system in which a code segment replicas. This synchronization capability is inherently multi presented in an intermediate language (e.g. Microsoft IL, master. US 7,743,019 B2 53 54 The synchronization capability of the present invention For every Change Unit, the synchronization service may be allows replicas to: able to determine whether it changed or did not change since determine which changes another replica is aware of the last Sync. request information about changes that this replica is not Designating Change Units in the schema serves several aware of purposes. First, it determines how chatty the synchronization convey information about changes that the other replica is service is on the wire. When a change is made inside a Change not aware of Unit, the entire Change Unit is sent to the other replicas, since determine when two changes are in conflict with each the synchronization service does not know which part of the other; Change Unit was changed. Second, it determines the granu apply changes locally: 10 larity of conflict detection. When two concurrent changes convey conflict resolutions to other replicas to ensure con (these terms are defined in detail in Subsequent sections) are Vergence; and made to the same Change Unit, the synchronization service resolve the conflicts based on specified policies for conflict raises a conflict; on the other hand, if concurrent changes are resolutions. made to different Change Units, then no conflict is raised and 1. Storage-Platform-to-Storage-Platform Synchronization 15 the changes are automatically merged. Third, it strongly The primary application of the synchronization service 330 affects the amount of metadata kept by the system. Much of of the storage platform of the present invention is to synchro the synchronization service metadata is kept per-Change nize multiple instances of the storage platform (each with its Unit; thus, making Change Units Smaller increases the over own data store). The synchronization service operates at the head of sync. level of the storage platform schemas (rather than the under Defining Change Units requires finding the right trade lying tables of the database engine 314). Thus, for example, offs. For that reason, the synchronization service allows "Scopes' are used to define synchronization sets as discussed schema designers to participate in the process. below. In one embodiment, the synchronization service does not The synchronization service operates on the principle of Support Change Units that are larger than an element. How “net changes'. Rather than recording and sending individual 25 ever, it does Support the ability for schema designers to operations (such as with transactional replication), the Syn specify Smaller Change Units than an element—namely, chronization service sends the end-result of those operations, grouping multiple attributes of an element into a separate thus often consolidating the results of multiple operations Change Unit. In that embodiment, this is accomplished using into a single resulting change. the following syntax: The synchronization service does not in general respect 30 transaction boundaries. In other words, if two changes are made to a storage platform data store in a single transaction, replicas atomically—one may show up without the other. The the same Item in the same transaction, then these changes are service. and initiate a sync operation. Such an application provides all of the parameters needed to perform synchronization (see

ReplicaKnowledge remoteKnowledge = ...; 35 FOO TransformRemoteToLocal (remoteChange, person); // Initialize ReplicaSynchronizer ctX.Update(); ctX. ReplicaSynchronizer = new ReplicaSynchronizer(replicaItemId, if Save the new remote anchor (this can be an extension on remotePartnerId); the replica) ctX.ReplicaSynchronizer. RemoteKnowledge = remoteKnowledge; ChangeReader reader = ctX. ReplicaSynchronizer. GetChangeReader(); 40 FOO SaveRemoteAnchor(); if Enumerate changes and process them // This is a regular WinFS APIsave since remote knowledge is not synced. bool bohangesToRead = true; while (bChangesToRead) remoteKnowledge = { 45 ctX. ReplicaSynchronizer. GetUpdatedRemoteKnowledge(); ChangeCollections.object> changes = null; ctX.Update(); bChangesToRead = reader. ReadChanges( 10, out changes); foreach (object change in changes) ctX.Close(); // Process enumerated object, adapter does it's own if Adapter callback for processing application status callbacks Schema transform 50 if and ID mapping. It may even retrieve additional void FOO OnEntitySaved.( object sender, ChangeStatusEventArgs objects from the args) if CtX for this purpose and modify adapter metadata after change remote Anchor. AcceptChange(args.ChangeStatus); fi has been applied to remote store ChangeStatus status = 55 FOOProcess.AndApplyToRemoteStore(change); // Update learned knowledge with status reader. AcknowledgeChange (changeStatus); 4. Methods of API Synchronization In one embodiment of the present invention, synchroniza tion between a WinFS store and a non-WinFS store is accom remoteKnowledge = ctX. ReplicaSynchronizer. GetUpdatedRemoteKnowledge(); 60 plished is possible via the Synchronization APIs exposed by reader. Close(); the WinFS-based hardware/software interface system. if Save updated knowledge and adapter metadata, if any In one embodiment, all synchronization adapters are required to implement the synchronization adapter API, a ctX.Update(); 65 common language runtime (CLR) managed API, so that they if Sample for change application, first initialize remote knowledge can be consistently deployed, initialized, and controlled. The adapter API provides: US 7,743,019 B2 67 68 A standard mechanism to register adapters with the hard may result and lead to additional future conflicts, the ware/software interface system synchronization frame incremental and sequential enumeration of conflicts as work. reflected in updated state information is invisible to A standard mechanism for adapters to declare their capa other replicas that receive that updated state informa bilities and the type of configuration information needed 5 tion. to initialize the adapter. At the root of the sync schema is the replica which has a A standard mechanism for passing initialization informa base type to define a root folder (in fact, a root Item) that tion to the adapter. has a unique ID, an ID for the sync community in which A mechanism for adapters to report progress status back to it is a member, and whatever filters and other elements the applications invoking synchronization. 10 are necessary or desireable for the specific replica. A mechanism to report any errors that occur during syn Each replica's "mapping is maintained within the replica chronization. and, as such, the mapping for any particular replica is A mechanism to request cancellation of an ongoing syn limited to the other replicas such replica knows about. chronization operation. While this mapping may only comprise a Subset of the There are two potential process models for adapters, 15 entire sync community, changes to said replica will still depending on the requirements of the scenario. The adapter propogate to the entire sync community via commonly can execute in the same process space as the application shared replicas (although any particular replica is invoking it or in a separate process all by itself. To execute in unaware of which other replicas it is commonly sharing its own separate process, the adapter defines its own factory with an unknown replica). class, which is used to instantiate the adapter. The factory can The sync schema includes both a plurality of predefined return an instance of the adapter in the same process as the conflict handlers available to all replicas, as well as the invoking application, or return a remote instance of the ability for user/developer defined custom conflict han adapter in a different Microsoft common language runtime dlers. The schema also may also include three special application domain or process. A default factory implemen “conflict resolvers': (a) a conflict “filter which resolves tation is provided which instantiates the adapter in the same 25 different conflicts in different ways based, e.g., (i) how process. In practice, many adapters will run in the same to handle when same change unit changed in two places, process as the invoking application. The out of process model (ii) how to handle when a change unit is changed in one is usually required for one or both of the following reasons: place but deleted in another; and (iii) how to handle Security purposes. The adapter must run in the process when two different change units have the same name in space of a certain process or service. 30 two different locations; (b) conflict “handler list where The adapter has to process requests from other sources— each element of the list specifies a series of actions to for example, incoming network requests in addition to attempt in order until the conflict is successfully processing requests from invoking applications. resolved; and (c) a "do-nothing log that tracks the con Referring to FIG.37, one embodiment of the present inven flict but takes no further action without user interven tion presumes a simple adapter that is unaware of how state is 35 tion. calculated or it associated metadata is exchanged. In this The sync schema and use of replicas enables a true distrib embodiment, synchronization is achieved by the replica, in uted peer-to-peer mutli-master synchronization com regard to the data source with which it wants to synchronize, munity. Moreover, there is no sync community type, but by first, at step 3702, determining which changes have the Sync community exists simply as a value in the occurred since it last synchronized with said data source, and 40 community field of the replicas themselves. the replica then transmits the incremental changes that have Every replica has its own metadata for tracking incremen occurred since this last synchronization based on its present tal change enumeration and storing state information for state information, and this present state information and the other replicas that are known in the sync community. incremental changes are to the data source via the adapter. At Change units have their own metadata comprising: a ver step 3704, the adapter, upon receiving the change data from 45 sion comprising a partner key plus a partner change the replica in the previous step, implements as many changes number; an Item/Extension/Relationship versioning for to the data source as possible, tracks which changes are Suc each change unit, Knowledge regarding the changes a cessful and which fail, and transmits the Success-and-failure replica has seen/received from the Sync community; a info back to WinFS (of the replica). The hardware/software GUID and Local ID configuration; and a GUID stored interface system of the replica (WinFS), at step 3706, upon 50 on a reference relationship for cleanup. receiving the Success-and-failure info from the replica, then calculates the new state information for the data source, stores IV. CONCLUSION this information for future use by its replica, and transmits this new state info back to the data source, that is, to the As the foregoing illustrates, the present invention is adapter for storage and Subsequent use by the adapter. 55 directed to a storage platform for organizing, searching, and D. Additional Aspects of the SYNC Schema sharing data. The storage platform of the present invention The following are additional (or more specific) aspects of extends and broadens the concept of data storage beyond the synchronization schema for various embodiments of the existing file systems and database systems, and is designed to present invention. be the store for all types of data, including structured, non Each replica is a defined synchronization Subset of data 60 structured, or semi-structured data, Such as relational (tabu from the entirety of a data store—a slice of data having lar) data, XML, and a new form of data called Items. Through multiple instances. its common storage foundation and Schematized data, the Conflict resolution policies are handled by each replica storage platform of the present invention enables more effi (and adaptor/data source combination) individually— cient application development for consumers, knowledge that is, each replica is able to resolve conflicts based on 65 workers and enterprises. It offers a rich and extensible appli its own criteria and conflict resolution schema. More cation programming interface that not only makes available ove, while differences in each instance of the data store the capabilities inherent in its data model, but also embraces US 7,743,019 B2 69 70 and extends existing file systemand database access methods. the synchronization Subsystem further configured to deter It is understood that changes may be made to the embodi mine which files stored in the non-synch-aware remote ments described above without departing from the broad computer system need updating based on the compari inventive concepts thereof. Accordingly, the present inven SOn, tion is not limited to the particular embodiments disclosed, the synchronization Subsystem further configured to trans but is intended to cover all modifications that are within the late items into files formatted for storage in the non spirit and scope of the invention as defined by the appended synch-aware remote computer system, claims. the synchronization Subsystem further configured to send As is apparent from the above, all orportions of the various the files formatted for storage in the non-synch to the systems, methods, and aspects of the present invention may 10 remote computer system, and be embodied in the form of program code (i.e., instructions). the synchronization Subsystem further configured to This program code may be stored on a computer-readable update the state information for files stored in the non medium, Such as a magnetic, electrical, or optical storage synch-aware remote computer system, wherein the non medium, including without limitation a floppy diskette, CD synch-aware remote computer system is configured to ROM, CD-RW, DVD-ROM, DVD-RAM, magnetic tape, 15 receive the files and store them in a file system. flash memory, hard disk drive, or any other machine-readable 2. The system of claim 1 wherein the synchronization storage medium, wherein, when the program code is loaded Subsystem synchronizes only a Subset of data, from among into and executed by a machine, such as a computer or server, the entirety of data on said data store, during a synchroniza the machine becomes an apparatus for practicing the inven tion operation. tion. The present invention may also be embodied in the form 3. The system of claim 1 wherein instructions that effectu of program code that is transmitted over some transmission ate the database management program that encapsulates the medium, Such as over electrical wiring or cabling, through file system are configured to execute during kernel mode. fiber optics, over a network, including the Internet or an 4. The system of claim 3 wherein the operating system intranet, or via any otherform of transmission, wherein, when further includes an application program interface that is con the program code is received and loaded into and executed by 25 figured to execute during kernel mode, wherein the applica a machine, such as a computer, the machine becomes an tion program interface is configured to expose the items apparatus for practicing the invention. When implemented on stored in the database management program to applications a general-purpose processor, the program code combines executing in user mode of the operating system. with the processor to provide a unique apparatus that operates 5. The system of claim 1 wherein the synchronization analogously to specific logic circuits. 30 Subsystem is configured to synchronize changes independent of the remote computer system. 6. The system of claim 1 wherein conflicts in synchroniza What is claimed: tion are automatically detected and resolved based on pre 1. A storage platform system including a processor and a defined determinable criteria. computer readable storage medium, said storage system com 35 7. The system of claim 6 wherein certain of said conflicts prising: are resolved by being logged for manual resolution by an end-user. instructions for an operating system, the operating system 8. The system of claim 1 wherein the synchronization including a kernel, wherein the kernel includes a data Subsystem tracks the state of previous synchronizations with base management program that encapsulates a file sys 40 a sync partner, and thereby only synchronizes change units tem, the database management program that encapsu with that partner that have changed since the last synchroni lates with the file system configured to store data in the Zation. file system as file streams, generate items that include 9. A method for synchronizing data stored in a computer metadata for the file streams and store the items in the system, said method comprising: database management program, the database manage 45 executing an operating system that includes a kernel, the ment program including a base schema and a mecha kernel including a database management program that nism configured to extend the base schema to define a encapsulates a file system, wherein the database man Schema for the data, and divide the data into program agement program includes a base schema and a mecha mably defined change units based on the schema for the nism to extend the base schema to define a schema for data, wherein a change unit is a smallest piece of schema 50 data; that is individually tracked by the database management storing, by the database management program that encap program integrated with the file system and the size of a sulates the file system, data in the file system as file change unit is adjustable; Streams; wherein the database management system is configured to generating, by the database management program that generate and store state information for files Stored in a 55 encapsulates the file system, items that include metadata non-synch-aware remote computer system, wherein the for the file streams, wherein the metadata is defined by state information includes change information for the the schema for the data; files stored in the non-synch-aware remote computer storing the items in the database management program; system, wherein the non-synch-aware remote computer dividing, by the database management program that encap system does not include a synchronization Subsystem 60 Sulates the file system, said data into programmably that is synchronization compatible with a synchroniza defined change units based on the schema for the data, tion Subsystem executing on the processor; wherein a change unit is a smallest piece of schema that the operating system further including the synchronization is individually tracked by the database management pro Subsystem configured to compare status information for gram that encapsulates the file system and the size of a items stored in the database management program to 65 change unit is adjustable; state information for corresponding files stored in the generating, by the database management program, state non-synch-aware remote computer system, information for files stored in a non-synch-aware remote US 7,743,019 B2 71 72 computer system, wherein the State information comparing, by the database management program, change includes change information for the files Stored in a unit information for items stored in the database man non-synch-aware remote computer system, wherein the agement program to state information for corresponding non-synch-aware remote computer system does not files stored in a non-synch-aware remote computer sys include a synchronization Subsystem that is synchroni tem; Zation compatible with the database management pro determining, by the database management program, which gram, items in the database management program need updat comparing, by the database management program, change ing based on the comparison; unit information for items stored in the database man receiving files corresponding to the items that need updat agement program to state information for corresponding 10 ing from the non-synch-aware remote computer system; files stored in a non-synch-aware remote computer sys translating, by the database management program, files tem; corresponding to the items that need updating into items; determining, by the database management program, which storing, by the database management program, the items files stored in the non-synch-aware remote computer obtained from the files corresponding to the items that system need updating based on the comparison; 15 need updating in the database management program; translating, by the database management program, items storing, by the database management program, updated into files formatted for storage in the non-synch-aware state information for files stored in a non-synch-aware remote computer system; remote computer system. sending, by the database management program, the files 14. The method of claim 13, further comprising: formatted for storage in the non-synch-aware remote calculating a new state of the information stored in the computer system to the remote computer system; and database management program based on the Success or updating, by the database management program, the state failure for each change on a change unit by change unit information for files stored in the non-synch-aware basis, storing the new state information. remote computer system, wherein the non-synch-aware 15. The method of claim 13, further comprising: remote computer system is configured to receive the files 25 receiving information that reflects whether each change and store them in a file system. was successful from the non-synch-aware remote com 10. The method of claim 9, wherein code that effectuates puter system; and the database management program that encapsulates the file storing, by the database management program that encap system is configured to execute during kernel mode of the Sulates the file system, said information. operating system. 30 16. A computer-readable storage medium comprising 11. The method of claim 10 further comprising detecting computer-readable instructions, the computer readable stor synchronization conflicts at the level of change unit granular age medium comprising: ity. instructions for an operating system, the operating system 12. The method of claim 10, further comprising: including a kernel and a shell, wherein applications instances reporting Success, failure, and/or conflicts at 35 execute on the shell of the operating system; individual change unit level on change application, the the instructions for the operating system including a data instances comprising sync data; and base management program that encapsulates a file sys applications using sync data for updating a backend state. tem, wherein the database management program that 13. A method for synchronizing data, said method com encapsulates the file system is part of the kernel of the prising: 40 operating system; executing an operating system that includes a kernel, the the instructions for the database management program that kernel including a database management program that encapsulates the file system configured to store data as encapsulates a file system; file streams in the file system and generate items for the storing, by the database management program that encap 45 file streams that include metadata for the file streams, sulates the file system, data in the file system as file wherein the format of the metadata conforms to one of a Streams; plurality of Schemas, each schema of the plurality is generating, by the database management program that derived from a base schema, further wherein each encapsulates the file system, items associated with the Schema of the plurality includes programmably defined file streams that include metadata for the file streams, 50 change units, whereina change unit is a smallest piece of wherein the metadata is defined by the schema for the schema that is individually tracked by the database man data, further wherein the items conform to a schema agement program that encapsulates the file system; derived from a base schema, the schema defining a size instructions for generating state information for files stored of a change unit, further wherein a change unit is the in a non-synch-aware remote computer system, wherein smallest piece of schema that is individually tracked by 55 the state information includes change information for the database management program that encapsulates the the files stored in a non-synch-aware remote computer file system; system, wherein the non-synch-aware remote computer storing the items in the database management program; system does not include a synchronization Subsystem generating, by the database management program, state that is synchronization compatible with the database information for files stored in a non-synch-aware remote 60 management program; computer system, wherein the State information instructions for comparing change unit information for includes change information for the files Stored in a items stored in the database management program to non-synch-aware remote computer system, wherein the state information for corresponding files stored in a non non-synch-aware remote computer system does not synch-aware remote computer system; include a synchronization Subsystem that is synchroni 65 instructions for determining which files stored in the non Zation compatible with the database management pro synch-aware remote computer system need updating gram, based on the comparison; US 7,743,019 B2 73 74 instructions for translating items into files formatted for information for the files Stored in a non-synch-aware storage in the non-synch- aware remote computer sys remote computer system, wherein the non-synch-aware tem; remote computer system does not include a synchroni instructions for sending the files formatted for storage in Zation Subsystem that is synchronization compatible the non-synch-aware computer system to the remote with the database management program; computer system; and the instructions for the database management program instructions for updating the state information for files including instructions for comparing change unit infor stored in the non-synch-aware remote computer system, mation for items stored in the database management wherein the non-synch-aware remote computer system program to state information for corresponding files is configured to receive the files and store them in a file 10 stored in a non-synch-aware remote computer system; system. the instructions for the database management program 17. The computer-readable storage medium of claim 16, including instructions for determining which files stored further comprising: in the non-synch-aware remote computer system need instructions for determining whether the non-synch-aware updating based on the comparison; computer system Successfully stored the files in a file 15 the instructions for the database management program system. including instructions for translating items into files for 18. The computer-readable storage medium of claim 16, matted for storage in the non-synch-aware remote com wherein the instructions for the database management pro puter system; gram that encapsulates the file system are configured to the instructions for the database management program execute during kernel mode of the operating system. including instructions for sending the files formatted for 19. The computer-readable storage medium of claim 16, storage in the non-synch-aware remote computer system wherein conflicts in Synchronization are automatically to the remote computer system; and detected and resolved based on predefined determinable cri the instructions for the database management program teria. including instructions for updating the state information 20. The computer-readable-storage medium of claim 19, 25 for files stored in the non-synch-aware remote computer wherein certain of said conflicts are resolved by being logged system, wherein the non-synch-aware remote computer for manual resolution by an end-user. system is configured to receive the files and store them in 21. The computer-readable storage medium of claim 16, a file system. wherein the synchronization process is configured to track the 23. The computer-readable storage medium of claim 22, state of previous synchronizations with a sync partner, and 30 wherein the instructions for the database management pro thereby only synchronizes change units with that partner that gram that encapsulates the file system are configured to have changed since the last synchronization. execute during kernel mode of the operating system. 22. A computer-readable storage medium comprising 24. The computer-readable storage medium of claim 23, computer-readable instructions for synchronizing data, said further comprising detecting synchronization conflicts at the computer-readable storage medium comprising: 35 level of change unit granularity. instructions for executing an operating system that include 25. The computer-readable storage medium of claim 23, instructions for a kernel, the instructions for the kernel further comprising: including instructions for a file system encapsulated by instances reporting Success, failure, and/or conflicts at a database management program, wherein the instruc individual change unit level on change application, the tions for the database management program include 40 instances comprising sync data; and instructions for a base schema and a mechanism to applications using sync data for updating a backend state. extend the base schema to define a schema for data; 26. A computer-readable storage medium comprising the instructions for the kernel of the operating system computer readable instructions for synchronizing data, said including instructions for storing data in the file system computer-readable storage medium comprising: as file streams; 45 instructions for executing an operating system that the instructions for the kernel of the operating system includes a kernel, the kernel including a file system including instructions for generating items that include encapsulated by a database management program; metadata for the file streams, wherein the metadata is the instructions for the kernel of the operating system defined by the schema for the data; including instructions for storing data in the file system the instructions for the kernel of the operating system 50 as file streams; including instructions for instructions for storing the the instructions for the kernel of the operating system items in the database management program; including instructions for generating items associated the instructions for the kernel of the operating system with the file streams that include metadata for the file including instructions for dividing said data into pro streams, wherein the metadata is defined by the schema grammably defined change units based on the schema 55 for the data, further wherein the items conform to a for the data, wherein a change unit is a smallest piece of Schema derived from a base schema, the schema defin schema that is individually tracked by the database man ing a size of a change unit, further wherein a change unit agement program that encapsulates the file system and is the smallest piece of schema that is individually the size of a change unit is adjustable; tracked by the database management program integrated the instructions for the database management program 60 with the file system; including instructions for sequentially enumerating instructions for storing the items in the database manage changes to said data and tracking said changes on a per ment program; change unit basis; instructions for generating state information for files stored the instructions for the database management program in a non-synch-aware remote computer system, wherein including instructions for generating state information 65 the state information includes change information for for files Stored in a non-synch-aware remote computer the files stored in a non-synch-aware remote computer system, wherein the state information includes change system, wherein the non-synch-aware remote computer US 7,743,019 B2 75 76 system does not include a synchronization Subsystem instructions for storing updated State information for files that is compatible with the database management pro stored in a non-synch-aware remote computer system. gram, 27. The computer-readable storage medium of claim 26, instructions for comparing change unit information for further comprising: items stored in the database management program to 5 state information for corresponding files stored in a non instructions for determining whether the non-synch-aware synch-aware remote computer system; computer system Successfully stored the files in a file instructions for determining which items in the database system. management program need updating based on the com 28. The computer-readable storage medium of claim 26, parison; 10 further comprising: instructions for receiving files corresponding to the items that need updating from the non-synch-aware remote instructions for receiving state information that reflects computer system; whether each change was successful from the non instructions for translating files corresponding to the items synch-aware remote computer system; and that need updating into items; 15 instructions for storing the information. instructions for storing the items obtained from the files corresponding to the items that need updating in the database management program;