The Importance of Data Models in Enterprise Metadata Management
Total Page:16
File Type:pdf, Size:1020Kb
THE IMPORTANCE OF DATA MODELS IN ENTERPRISE METADATA MANAGEMENT October 19, 2017 © 2016 ASG Technologies Group, Inc. All rights reserved HAPPINESS IS… SOURCE: 10/18/2017 TODAY SHOW – “THE BLUE ZONE OF HAPPINESS” – DAN BUETTNER • 3 Close Friends • Get a Dog • Good Light • Get Religion • Get Married…. Stay Married • Volunteer • “Money will buy you Happiness… well, it’s more about Financial Security” • “Our Data Models are now incorporated within our corporate metadata repository” © 2016 ASG Technologies Group, Inc. All rights reserved 3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES 1. Happiness is individually sought and achievable © 2016 ASG Technologies Group, Inc. All rights reserved 3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES 1. Happiness is individually sought and achievable 2. Data Models can in be incorporated into your corporate metadata repository © 2016 ASG Technologies Group, Inc. All rights reserved 3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES 1. Happiness is individually sought and achievable 2. Data Models can in be incorporated into your corporate metadata repository 3. ASG Technologies can provide an overall solution with services to accomplish #2 above and more for your company. © 2016 ASG Technologies Group, Inc. All rights reserved AGENDA Data model imports to the metadata collection View and search capabilities Traceability of physical and logical models End-to-End Data Lineage, impact analysis Metadata workflow and lifecycle management © 2016 ASG Technologies Group, Inc. All rights reserved DATA MODELING LANDSCAPE DATA MODELING TO ENTERPRISE METADATA MANAGEMENT – (EMM) DATA MODEL TOOLS DATA MODEL CAPABILITIES EMM • ERwin • Forward / Reverse • Metadata Repository Engineering – Data Model • IDERA Er-Studio – Business Glossary • Mapping – Reference Data • IBM InfoSphere Data – Conceptual Architect – Logical – Physical • Oracle SQL Developer DM • End-to-End Data Lineage • Data Dictionary Standards • SAP Power Designer • More … • Sparx Enterprise Architect • Others … © 2016 ASG Technologies Group, Inc. All rights reserved DATA MODELING LANDSCAPE BASF – “WE DON’T MAKE THE WATER, WE MAKE IT CLEANER” ASG – “WE DON’T MAKE THE DATA MODEL, WE INCORPORATE IT WITHIN YOUR METADATA REPOSITORY” DATA MODEL TOOLS DATA MODEL CAPABILITIES EMM • ERwin • Forward / Reverse • Metadata Repository Engineering – Data Model • IDERA Er-Studio – Business Glossary • Mapping – Reference Data • IBM InfoSphere Data – Conceptual Architect – Logical – Physical • Oracle SQL Developer DM • End-to-End Data Lineage • Data Dictionary Standards • SAP Power Designer • More … • Sparx Enterprise Architect • Others … © 2016 ASG Technologies Group, Inc. All rights reserved Populating the Metadata Repository Could be manually input for the content on the left …. • Business Glossary • Data Models – Glossary, Contexts, Terms – Entities, attributes, relationships, … • Business Packages • Relational Databases – Business Rules, Processes, Critical Data – Tables, Columns, indexes, ... Elements, Policies, Standards • Transformations • Business Applications – Task, Maps, Transformations, … – Application • Business Intelligence • Reference Data – Reports, Cubes, Fields, Maps, … – Contexts, Concepts, Values, Meanings, Permissible Values, Hierarchies • Data Structures – Files, fields, … • Big Data © 2016 ASG Technologies Group, Inc. – Tables, Files, Streams, … All rights reserved Populating the Metadata Repository Could be manually input for the content on the left …. But not for the content on the right !! • Business Glossary • Data Models – Glossary, Contexts, Terms – Entities, attributes, relationships, … • Business Packages • Relational Databases – Business Rules, Processes, Critical Data – Tables, Columns, indexes, ... Elements, Policies, Standards • Transformations • Business Applications – Task, Maps, Transformations, … – Application • Business Intelligence • Reference Data – Reports, Cubes, Fields, Maps, … – Contexts, Concepts, Values, Meanings, Permissible Values, Hierarchies • Data Structures – Files, fields, … • Big Data © 2016 ASG Technologies Group, Inc. – Tables, Files, Streams, … All rights reserved Metadata Collection / Import Metadata Collection Architecture – Modelling Bus – (Bi Directional) Enterprise Architecture Modelling Rochade Application Import • SAG Predict • ARIS Toolset Export Determine • Borland Convert to Existing • CaliberRM Item Names Import File Standard Items • Together Control/Center Format • Casewise Corporate Modeler • COOL:Biz ; COOL:gen Names File • PowerDesigner • ER/Studio • ERwin • IBM/Rational Rochade Compare • Data Architect File • Software Modeler (RSM) Tool File • Software Architect (RSA) • System Architect • Systems Developer • Rose • Rational Rose XDE • mictroTool case/4/0 © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Collection / Import Metadata Collection Architecture – Application Model – Physical and Logical Layer - Process • Scan in the models • Define rules and process • Conceptual and Logical entity names should match. • There has to be a link between physical table with the logical entity in the application model (as shown in figure) • Custom work to provide audit reports, dashboards etc. For example: • Find database tables which are based on physical model. • Find physical model entities which are not implemented in the database. • Find missing links/mismatches the enterprise model and application models • Standard model linker will link model to physical data. • Custom linker is required to create the link between conceptual and logical/physical models. © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence Browser – View © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence Browser – View © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence Browser – Search © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence Browser – Search - Account © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence MyInfoAssist – Search – Account – Logical and Physical views © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence Traceability – Logical and Physical views © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence Traceability – Logical Model Reporting © 2016 ASG Technologies Group, Inc. All rights reserved IMPLEMENTING DATA LINEAGE ID Critical Data Scan Lineage Automate Lineage We build out a reverse tracing methodology and base line for comprehensive and accurate end to end data lineage. Business Glossary Business Traceability Change Detection © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence End to End Zero Gap Data Lineage – Account – Business application view © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence End to End Zero Gap Data Lineage – Account – High Level view with Impact Analysis © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence End to End Zero Gap Data Lineage – Account – Detailed Level view with ETL and Impact Analysis © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence End to End Data Lineage – Account – Detailed Level view – Impact Analysis and Visualization © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence End to End Data Lineage – SnapShot – Lineage Comparison – Impact Analysis and Visualization 1) Current WIP Lineage 2) Click to Compare WIP changes with Prod (high level) within the Rochade Browser 3) WIP/Production compare at a column level Legend: Grey=same in both Red=Deleted Green=Added © 2016 ASG Technologies Group, Inc. All rights reserved Metadata Repository Data Intelligence Metadata Workflow and lifecycle management Governance Board Business & Data Stewards © 2016 ASG Technologies Group, Inc. All rights reserved ASG Enterprise Data Intelligence Core Capabilities New words here…. DATA GOVERNANCE Enterprise Metadata, Business modeling, CDE subscription, Workflow, 01 Issue management DATA LINEAGE Lineage Life Cycle, Snapshot, 02 Stitching, Impact Analysis, Data Lineage Appliance AUTO DATA COLLECTION Application discovery & 03 understanding via 230+ Scanners for structured & unstructured data DATA DISCOVERY 04 Global search, drill-down REGS & COMPLIANCE SUPPORT 05 Attestation documents and Reports © 2016 ASG Technologies Group, Inc. All rights reserved 27 SUMMARY DATA MODELING TO ENTERPRISE METADATA MANAGEMENT • Automated scanners or buses reduce complexity and increase accuracy • Look for one solution that can provide full EMM • Implementation Services will be key Training & Consulting Scanner Development Custom Development Services On-premise & SaaS Implementation Services Industry Leading Metadata Best Practices Services Packages SaaS Hosting & Maintenance Resources Metadata Practice © 2016 ASG Technologies Group, Inc. Experts All rights reserved 3 POINTS TO REMEMBER – QUIZ TIME SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES 1. Happiness is individually sought and achievable © 2016 ASG Technologies Group, Inc. All rights reserved 3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA