Creating the Golden Record
Total Page:16
File Type:pdf, Size:1020Kb
CreatingCreating thethe GoldenGolden RecordRecord BetterBetter DataData throughthrough ChemistryChemistry Donald J. Soulsby metaWright.com AgendaAgenda • The Golden Record • Master Data • Discovery • Integration • Quality • Master Data Strategy DAMADAMA –– LinkedInLinkedIn GroupGroup C. Lwanga Yonke - Information Quality Practitioner ......SpewakSpewak advocatedadvocated usingusing datadata dependencydependency toto determinedetermine thethe idealideal sequencesequence inin whichwhich applicationsapplications shouldshould bebe developeddeveloped andand implemented:implemented: “Develop“Develop thethe applicationsapplications thatthat createcreate datadata beforebefore thosethose thatthat needneed toto useuse thatthat data”data” (p.10).(p.10). ArchitectureArchitecture AdvocatesAdvocates WilliamWilliam SmithSmith – Entity Lifecycle CliveClive FinkelsteinFinkelstein - Information Engineering – CRUD RonRon RossRoss -- ResourceResource LifeLife CycleCycle AnalysisAnalysis CRUDCRUD inin aa PerfectPerfect WorldWorld CanonicalCanonical SynthesisSynthesis Broadly speaking, materials scientists investigate two types of phenomena. Both are based on the microstructures of materials: … ii. How do these microstructures influence the properties of the material (such as strength, electrical conductivity, or high frequency electromagnetic absorption)? http://www.its.caltech.edu/~matsci/WhatIs2.html Business VS Development Life Cycles Zachman Framework for Enterprise Architecture WHERE WHAT WHEN HOW WHO WHY CONTEXTUAL List of List of List of List of List of Scope List of things Processes Locations Organization Events Business Units Goals/Stat. CONCEPTUAL Business Business Process Logistics Work Flow Master Business Business Model Entity Model Model Network Model Schedule Plan Application System Human LOGICAL Logical Data Process Network Interface Processing Business System Model Model Model Model Paradigm Structure Rule Model Application Network PHYSICAL Physical Structure Technology Presentation Control Rule Design Technology Model Data Model Chart Model Architecture Structure OUT-OF- Data Network Interface Timing Rule CONTEXT Definition Program Components Components Definition Specification Components PRODUCT DATABASE APPLICATION NETWORK ORGANIZATION SCHEDULE STRATEGY Functioning System TibetanTibetan ProverbProverb “ If upstream is dirty, downstream will be muddy” DataData WarehousingWarehousing -- ETLETL TransmutationTransmutation TheThe SecretSecret BernardBernard Trevisan,Trevisan, aa 15th15th centurycentury alchemist,alchemist, sspent much of his life and a sizable fortune in search of the secret of turning base metals into gold, realized while dying was… ““ToTo makemake gold,gold, oneone mustmust startstart withwith gold.”gold.” DAMADAMA GuideGuide toto thethe DataData ManagementManagement BodyBody ofof KnowledgeKnowledge Reference and Master Data Management: Managing golden versions and replicas DataData ManagementManagement BodyBody ofof KnowledgeKnowledge © 2008 DAMA International DAMADAMA DictionaryDictionary ReferenceReference DataData Any data used to categorize other data, or for relating data to information beyond the boundaries of the enterprise. See master data. MasterMaster DataData Synonymous with reference data. The data that provides the context for transaction data. It includes the details (definitions and identifiers) of internal and external objects involved in business transactions. Includes data about customers, products, employees, vendors, and controlled domains (code values). DAMADAMA DictionaryDictionary MasterMaster DataData ManagementManagement (MDM)(MDM) Processes that ensure that reference data is kept up to date and coordinated across an enterprise. The organization, management and distribution of corporately adjudicated data with widespread use in the organization. ReferenceReference && MasterMaster DataData ManagementManagement Ensuring consistency with a “golden version” of data values. One of nine data management functions identified in the DAMA-DMBOK Functional Framework. TopTop 1313 MDMMDM buzzwordsbuzzwords 1.1. MetadataMetadata (Don’t(Don’t saysay metadata)metadata) 2. Product information management (PIM) 3. Enterprise master patient index (EMPI) 4. Data governance 5. Customer data integration (CDI) 6. MDM hub 7. MDM architecture 8. "Collaborative" vs. "analytical" MDM 9. MDM return on investment (ROI) 10. MDM stakeholders 11. Enterprise hierarchy management 12. MDM metrics 13. Data competency center 02 Jul 2009 | Justin Aucoin, Assistant Site Editor, searchdatamanagement.com TheThe QuestQuest Bloor Research: the discovery of relationships between data elements, regardless of where the data is stored Discovery dDIQ MDM Quality Integration DAMA: The degree to which data DAMA: The planned and is accurate, complete, timely, controlled transformation and consistent with all requirements flow of data across databases, and business rules, and relevant for operational and/or analytical for a given use. use. AtomicAtomic DataData ofof thethe EnterpriseEnterprise Provide and maintain a consistent view of an enterprise's core business information assets Mendeleev ALCHEMY VS. CHEMISTRY WhatWhat isis MasterMaster Data?Data? Point In Time Currency Customer $ Product Country Transaction A Transaction is relationship of Master Data + Reference Files + time stamp + volumes + secondary descriptors MasterMaster DataData -- ClassificationClassification Customer Code Master AR Customer Data ID Cross ISO 3166 Reference INVENTORY Country Universal Data Code Enterprise Community Reference Status Data Code Local Operational Analytical Unified Master Data Types Operational Master Data Definition, creation, and synchronization of Master Data required for transactional systems and delivered via service-oriented architecture (SOA). Analytical Master Data Definition, creation, and integration, including multiple historical versions, of Master Data required for Enterprise Reporting or Data Warehousing applications. Unified Master Data Master Data that is defined and crated to apply to both Operational and Analytical applications. Implementation of the “single view of the truth”. It requires achieving agreement on a complex topic among a group of people MasterMaster DataData IntegrationIntegration Universal Master Data that is created and maintained by an external organization. For reference data it is often developed by a standards organization such as the ISO. Master Data Models are often available to members of an industry trade or professional association Enterprise Master Data that represents the common business information assets that need to be agreed on and shared throughout the enterprise Community Master Data that is shared between two or more applications. Typically data is replicated from the application that contains the system of record for the Master Data, by federation (key linkage) or propagation (materialized views). Local Master or Reference Data that is created and maintained for a single operational application. Much of the Reference Data will remain local, such as Status Codes, for the transactions within the application. MasterMaster DataData -- ContentContent Master Data Transactional Analytical Unified Definition Operational Technical Business Product Place Process Period Person Atomic Primitives PIM PRODUCT PROCESS PURPOSE PERIOD PEOPLEPARTY PLACE CDI MasterMaster DataData ClassificationClassification Artifact Person Product Place Period Process (Who) (What) (Where) (When) (How) Internal Product Time Business Geography Organization Hierarchy Slice Activity Customer Service Geo-Political Seasonality Go To Market MDMMDM –– IntegrationIntegration TechniquesTechniques DataData PropagationPropagation (Today)(Today) Copies Master Data from one location to another (materialized view). DataData ConsolidationConsolidation Captures data from multiple data sources and integrates into a single Master Data hub. DataData FederationFederation Provides a single virtual Master Data view of one or more data sources. No data is stored in the Master Data hub. VS ArchitectureArchitecture choiceschoices Independent Data Mart Bus Data Mart Architecture R e f F i l e Federated Data Stores Hub & Spoke Centralized Architecture Data Warehouse (Corporate Information Factory) (Enterprise Data Warehouse) Metadata + Repository Are master data files metadata? Need for central repository Inside Metadata Outside Metadata Repository Repository Metadata Repository Metadata Repository Master Data Master Data Meta Data MasterMaster DataData ArchitectureArchitecture –– RepositoryRepository • Comprehensive, single source for Master Data • Centrally managed, highly effective data governance • Fixed data model – typically Customer & Product • Extensive and sophisticated data integration process • All applications need to change to conform to Hubs • Expensive to modify to new business requirements • Hard to adapt to rapid changes S Legacy Data E Source Product R Application V I Customer C New Data HUB E Application Source S MasterMaster DataData ArchitectureArchitecture –– RegistryRegistry • Linkages to other data stores with transform logic • Low Cost, small footprint, only unique key kept in DB • Minimal disruption of current applications • Limited Data Governance, no merged Master Data • No resolution of semantic disintegrity • No History, System of Record linked to Data Source • Query performance - data availability from multiple sources Application Data Source MDM New Application XREF Application Data Source MasterMaster DataData ArchitectureArchitecture