PROFESSIONAL PETROLEUM ASSOCIATION

Introduction to Master Data Management

Master Data Management Integration with PPDM 3.8 Data Management with PPDM 3.8

1 Copyright 2013, PPDM Association. All Rights Reserved LEARNING OBJECTIVES

Understand and discuss Effective Data Management Strategies Understand and discuss Data Governance Understand and discuss Master Data Management Understand and discuss Data Quality Understand and discuss Data Organization and Architecture Understand how to use PPDM 3.8 for MDM

2 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

How Does Data Management Fit into Corporate Life?

Mainstream vs “burden”

3 Copyright 2013, PPDM Association. All Rights Reserved EFFECTIVE DATA MANAGEMENT STRATEGIES

• Effective data management is essentially being able to deliver the data that people need, when they need it, and in the format that they need it • Any data management strategy needs to start with the needs of the business and the key decisions that they make • What information is used to make the decisions? • What data is used to derive that information? • This can then be used to assess the value of data: • Who uses it and what do they use it for? • What impact does it have on business decisions? • This, in turn, will drive decisions around governance, architecture, organization, quality, and security.

4 Copyright 2013, PPDM Association. All Rights Reserved EFFECTIVE DATA MANAGEMENT STRATEGIES

5 Copyright 2013, PPDM Association. All Rights Reserved EFFECTIVE DATA MANAGEMENT STRATEGIES

6 Copyright 2013, PPDM Association. All Rights Reserved EFFECTIVE DATA MANAGEMENT STRATEGIES

7 Copyright 2013, PPDM Association. All Rights Reserved EFFECTIVE DATA MANAGEMENT STRATEGIES

Data Value

Data Master Data Governance Data Mgmt Quality

Data Data Data Organization Architecture Security

People Process

Technology

Effective Data Management

8 Copyright 2013, PPDM Association. All Rights Reserved ELEMENTS OF EFFECTIVE DATA MANAGEMENT STRATEGIES

Data Management Capabilities

Data Data Data Master Data Data Governance Organization Architecture & Quality

. Data Sizing, . Data Policies . Data Taxonomy Storage & . Data Quality Rules . Data Definitions . Data Standards . Logical Data Movement & Policies . Master Data . Business Data Models Architecture . Data Cleansing . Metadata Ownership . Business Process . Data Retention & Standards Process . Reference Data . Data Workflow Flows Deletion Policies . Compliance Rules . Phys Data Models . Data . Data Stewards . Solution Architects Administration . Business Data Enterprise Data . Data Quality . Business Data . . Storage/ Administration Owners Architects Technical Services Team . Data Stewards . Data Czar or Mgt. . Data Modelers Architects . Data Governance People . Data Owners Committee . DBAs . DBAs

. Archiving Tools . Master Data Mgt. . , Data Rules Library Data Modeling Tools . . . Storage Quality & . Automated Tools Management & . Reference Data

Monitoring Tools Notifications . Design/CASE Hardware Architecture . ETL Tools (Workflow) Tools . Technical . Metadata . Audit Reports Technology Architecture Repository

Copyright 2006 Accenture All Rights Reserved

9 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

Data Governance

10 Copyright 2013, PPDM Association. All Rights Reserved DATA GOVERNANCE

Data Governance is how an organization manages its data assets. • The business rules, policies, practices, procedures, roles, and responsibilities to create a consistent enterprise of an organization’s data • Data Governance is the foundation of effective data management but is too often neglected • It involves a great deal of work to establish and maintain a comprehensive Data Governance program • The PPDM 3.7/3.8 data management module provides a strong framework for Data Governance

11 Copyright 2013, PPDM Association. All Rights Reserved DATA GOVERNANCE

Legislation Shareholders PPP Policy Upper mgmt

Associations Practice Standards Mid mgmt

People Procedure Docs SW

12 Copyright 2013, PPDM Association. All Rights Reserved DATA GOVERNANCE

• POLICY – We need accountability each master data store to establish a focal point for responsibility. • PRACTICE – We must document what job functions are responsible for data creation, updates and deletes. • PRACTICE – We must document what software applications are responsible for data creation, updates and deletes. • PRACTICE – We must document where in the life cycle of a business asset it is created and used by various functional groups in our organization. • PROCEDURE – all replicated (or derived) copies will be derived from the master store, using an appropriate mechanism for replication management.

13 Copyright 2013, PPDM Association. All Rights Reserved DATA GOVERNANCE

• Business Rules describe or application procedures that are used to enforce Policies, Practices, and Procedures • These rules help ensure that the data within the database: • Conforms to convention in terms of meaning and loading practice • Is the most trusted data available; the source is known, the history is known, and the quality is consistent and understood • Is maintained consistently over its lifecycle

14 Copyright 2013, PPDM Association. All Rights Reserved BUSINESS RULES

Business rules • Create rules and associate with tables,

columns, procedures etc. PPDM_RULE • Rules enforcement methods R_PPDM_RULE_CLASS Kinds of business rules R_PPDM_RULE_PURPOSE • Policies, Practices, Procedures PPDM_RULE_XREF PPDM_RULE_DETAIL • Quality (trustworthiness) PPDM_RULE_COMPONENT

• Entitlements and security R_PPDM_XREF_TYPE

• Relationships PPDM_GROUP • Constraints (database and data) PPDM_RULE_ENFORCEMENT • Conversions or transformations R_PPDM_ENFORCE_METHOD R_PPDM_FAIL_RESULT • Calculations

15 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

Master Data Management

16 Copyright 2013, PPDM Association. All Rights Reserved WHAT DO WE HOPE TO ACHIEVE?

Any kind of data management is a lot of work.

Data management is not always recognized as a good value proposition by management.

- As a company - As an industry

17 Copyright 2013, PPDM Association. All Rights Reserved THE CHALLENGE TIP: SOURCE SYSTEMS LIE

• Source systems may not be able to accurately reflect the business! • Legacy systems • Assume out dated processes • May flatten or denormalize your data • May not account for regional variations • May not account for temporal variations • May not capture all the important information • May replicate information that they don’t “own” • …. • Just because data is in a legacy system does not mean you should load it into an MDM! • Ask lots of questions; ask everybody! • Don’t be afraid to recommend against loading certain data types!

19 Copyright 2013, PPDM Association. All Rights Reserved

COST VS BENEFIT

What is the cost of bad data? • Use the fear factor to engage management • Threat management succeeds in the short term – sometimes. • Does it really help you to avoid costly errors? What is the value of good data management? • Is data management really a “core” function? • Does it really save time and effort? • Can’t I just leave this to my vendors? Is data an asset? • How much is it worth? • Can data be put “on the books” as a tangible asset? - Will the IFRS help us?

20 Copyright 2013, PPDM Association. All Rights Reserved INTEGRATED ARCHITECTURE

Ontologies Meta Data Taxonomies Physical Spatial/GIS

PPDM

SOR MDM SOR

SOR

Business Rules Policies, Practices & Procedures VALUE OF INTEGRATED ARCHITECTURES

The asset value of core data is Life cycle based management recognised • You know who the “real owners” of • By the users data are • By the data managers Data flows can be identified and managed • By IT staff • Beware the “rogue” user! • By managers Maintenance effort should be Business rules can be applied reduced consistently Integration effort is probably higher! Key data is stored once, used many times! • Managed replication • May require translation, granularity shifts, equivalences …

22 Copyright 2013, PPDM Association. All Rights Reserved SOR ORDER FROM CHAOS

MDM / SOR SOR Warehouse

SOR = System of Record Gold Standard

MDM = Master Data Management

23 Copyright 2013, PPDM Association. All Rights Reserved MASTER DATA MANAGEMENT

Financial Data Physical Data

Spatial Data

HSE Data Unstructured / Semi-structured Data

Operational Structured Data Data 24 Copyright 2013, PPDM Association. All Rights Reserved AN OBJECTIVE: THE DIGITAL OILFIELD

001111000111011101100101011011000110110001011111010010010100010000111110001100010011001000110011001101000011010100110110001101 110011100000111001001100000011110000101111011101110110010101101100011011000101111101001001010001000011111000101100001111000101 001101010000010101010100010001011111010001000100000101010100010001010011111000110010001100000011000000110111010111110011000000 110110010111110011001000110111001111000010111101010011010100000101010101000100010111110100010001000001010101000100010100111110 1234567890,2007_06_27

25 Copyright 2013, PPDM Association. All Rights Reserved OBJECTIVES REALITY

Correct

Complete Current

26 Copyright 2013, PPDM Association. All Rights Reserved WORK SMARTER – NOT HARDER!

2005 study at Institute of Psychiatry • University of London IQ test with distraction is 10 points lower than with no distractions.

100

95

90

85

80 IQ test with distraction is No Smoking Pot Distractions Distractions 6 points lower than those who smoked marijuana. 27 Copyright 2013, PPDM Association. All Rights Reserved ABOUT KNOWLEDGE WORKERS

We have less productive time available University of California study of Knowledge Workers (in the US)

11 minutes between 2.1 hours lost per knowledge worker interruptions! per day to interruptions and recovery time

28,000,000,000 Costs in the US hours / year!! $588 Billion / year Gloria Marks, University of California at Irvine 28 Copyright 2013, PPDM Association. All Rights Reserved WHAT IS MASTER DATA MANAGEMENT (MDM)? Technical MDM • In computing, master data User Interface User Interfaces (Many) management (MDM) comprises a set of processes and tools which consistently define and manage the non-transactional data entities of an organization (also called reference Bus Analysis data). Bus Intel Collaborate • MDM has the objective of providing Decisions processes for collecting, aggregating, matching, consolidating, quality- Technical Life Cycles assuring, persisting and distributing Transact such data throughout an organization in such a way as to ensure Interpret consistency and control in the Master Data (One) ongoing maintenance and application use of this information. PPP • At a basic level, MDM seeks to Bus Rules ensure that an organization does not use multiple (potentially inconsistent) Quality versions of the same master data in Metrics different parts of its operations, which Ref data can occur in large organizations. Source Data (Many) Wikipedia 29 Copyright 2013, PPDM Association. All Rights Reserved WHAT IS MASTER DATA MANAGEMENT?

“Master data management (MDM) comprises a set of processes and tools which centrally and persistently define the non-transactional entities of an organization (also called Reference data). The objective of MDM is to collect from, and supply to various processes, unique instances of each entity.”

People

Technology Process

It’s People, Processes and Technology!

http://en.wikipedia.org/wiki/Master_Data_Management

30 Copyright 2013, PPDM Association. All Rights Reserved WHAT IS DATA GOVERNANCE?

“Data governance encompasses the people, processes and technology required to create a consistent, enterprise view of an organization's data in order to: • Increase consistency & confidence in decision making • Decrease the risk of regulatory fines • Improve data security People • Consistent information quality across the organization • Maximize the income generation potential of data • Designate accountability for information quality.”

Technology Process

It’s People, Processes and Technology! http://en.wikipedia.org/wiki/Data_governance

31 Copyright 2013, PPDM Association. All Rights Reserved WHAT’S THE DIFFERENCE?

Master data management keeps the data centralized, organized, unique

Data governance makes the data worth keeping!

Data governance can exist without MDM,

but the value of good MDM depends on good data governance! 32 Copyright 2013, PPDM Association. All Rights Reserved MASTER DATA MANAGEMENT ARCHITECTURES

Central (coexistence) master, where data coexists as managed replication from a source. Better data quality.

Registry, where the location and identity of the master is located, but most data is not replicated (meta data). Data issues not resolved.

Transaction hub, in which the master takes over the Sometimes underlying function of the source. systems are eliminated Highest data quality is possible.

33 Copyright 2013, PPDM Association. All Rights Reserved ALONG THE WAY

User Interfaces (Many)

Bus Analysis Bus Intel Collaborate Decisions Life Cycles

PPDM / MDM “Single Version of the Truth”

Data Governance Data Quality Business Rules

SOURCE SOURCE SOURCE A B C

34 Copyright 2013, PPDM Association. All Rights Reserved MAINTAINING MASTER DATA

Single Copy • Best but can be expensive • Requires a lot of software changes, vendors may not support Multiple Copies, Single Maintenance Master Data (One) • Data is updated in the master and disseminated to where it’s needed • SOA approach Continuous Merge • Data is updated in the source and propagated to the master for reconciliation and then propagation to other systems • Problems - Concurrent updates: What if two sources change the same entity – how do you deal with at the master level? Source Data (Many) - How do you integrate merged data back into the sources, once two entries have been harmonized? - Units of conversions – scale errors

35 Copyright 2013, PPDM Association. All Rights Reserved HOW MDM ARE USED

Collaboration Effective use of an MDM depends on • Create information once, use it many times your ability to define Operational controls (Consistency) content standards • Data flows through many life cycle transactional processes (SOA) • Data is born, modified, used, transformed …. Analytical systems • Identity analytics: Understand the objects critical to our business - What wells do I have an interest in • Analytics on master data: Improve operational efficiencies - Which wells have critical deadlines coming up? - Which pumps need to be changed out, so they don’t fail in the well bore? • Analytics integration - With warehouses, technical applications … - Improve the quality of data provided to other systems to improve their analytic capability

36 Copyright 2013, PPDM Association. All Rights Reserved USUAL COMPONENTS OF AN MDM (NOUNS)

People • Who are the people and organizations that we do business with? • Can I be unambiguous? Places • What is a country? (geographic, geopolitical) Things • What are the assets of the organization? • How do they relate to each other? Concepts • What contracts, interest sets, licenses are in place? • Am I managing these properly? http://msdn.microsoft.com/en-us/library/bb190163.aspx 37 Copyright 2013, PPDM Association. All Rights Reserved TO GET SEMANTIC CLARITY YOU MUST DEVELOP OR USE STANDARDS!

http://msdn.microsoft.com/en-us/library/bb190163.aspx 38 Copyright 2013, PPDM Association. All Rights Reserved

WHAT BELONGS IN THE MDM?

How long How itis important? of sharing common data? Value

How often and how much does it change? Where does your information sit?

http://msdn.microsoft.com/en-us/library/bb190163.aspx 39 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION What MDM Brings to your Information Architecture

Opportunities for improvement

40 Copyright 2013, PPDM Association. All Rights Reserved INTEGRATED COMMUNICATIONS

41 Copyright 2013, PPDM Association. All Rights Reserved DATA’S ASSET VALUE AND LIFE CYCLE

6 Planning Operations Administration 5

4

3 Value

2

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Time

42 Copyright 2013, PPDM Association. All Rights Reserved AGGREGATION HIERARCHIES

These may not exist in a source system!

Reporting Hierarchy Hierarchy Type (Specific) (General) Level 1 = COUNTRY CANADA

Level 2 = PROVINCE ALBERTA SASK MANITOBA

LEVEL 3 = FIELD Organizational RED GREEN BLUE ORANGE FIELD FEILD FIELD FIELD Hierarchies Resent Contributions to a Reporting 50% 85% 60% 50% 25% Hierarchy 50% 15% 40% 25%

RESENT A RESENT C RESENT E RESENT G RESENT I

RESENT B RESENT D RESENT F RESENT H

43 Copyright 2013, PPDM Association. All Rights Reserved THE IMPORTANCE OF VERSIONING

What did this data look like in the source? What rules did I use to harmonize my sources? • Granularity issues • Blending data If the value has changed, who made the change, and why? • Authentication • Provenance

Process standards are critical

44 Copyright 2013, PPDM Association. All Rights Reserved CONTENT – THE HARD BIT

Units of measure Organizational hierarchies Geographic areas identification and definition These are content standards! Well status • Corporate Products and substances • Industry • Global What is a well, anyway?

Company names and identities People and their roles

Create the set that works for the business and map the sources to that set Don’t force users to a single layer hierarchy – reality is a multidimensional business

45 Copyright 2013, PPDM Association. All Rights Reserved Phase 1 may be a small part of the LIFE CYCLE MANAGEMENT business

Acquire Land Rights Identify Opportunity Well Planning

Form Partnerships Interpretation

Drilling and Completion

Facility Exploration Data Management Management

Well Management

Seismic Exploration

Production Tracking

Decommission / Relinquish 46 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION Components of an MDM Strategy

47 Copyright 2013, PPDM Association. All Rights Reserved CHALLENGES AND COMPONENTS

People

Technology Process

48 Copyright 2013, PPDM Association. All Rights Reserved MDM CHALLENGES

Data ownership (who owns the data) Strategic vision (silo view master view)

Change management (how are changes to bad Performance management data propagated?) (fear, as MDM exposes People problems in process)

Data quality management (what is Transition planning (keep it, what are the rules?) working while you move forward) Where does bad data get fixed and how is the fix propagated?

Performance (speed of Technology Process queries, latency for data Data standards (semantics, what corrections to be happens when data that has been propagated) integrated has to be distributed back to the source? Meaning may be lost) What is the architecture going to be? Integration (how does the data move around?)

49 Copyright 2013, PPDM Association. All Rights Reserved PEOPLE COMPONENTS

In house users Project leaders Integrators Regulators

Data managers Software engineers Partners MDM team Governance leaders Public DBAs

Business users Computer Evangelists engineers IT Specialists People Network engineers

50 Copyright 2013, PPDM Association. All Rights Reserved PROCESS COMPONENTS

Business Design (the objectives)

Technical Design (the vision / plan) Define the process improvement goals Situation analysis Shewart Cycle / Deming Wheel Gaps Measure the current process Opportunities Process

Set Objectives and Priorities Analyze needs

Six Sigma Data collection (identify and harvest) Improve or optimize the Normalization and process reconciliation (reference codes)

Control to correct (mapping / business rules) variances

Data governance (process controls)

51 Copyright 2013, PPDM Association. All Rights Reserved TECHNOLOGY COMPONENTS

Flat files

Corporate Data and Information Data warehouses and Storage Datamarts

Operational data store

Data storage (hardware)

Technology

Business Intelligence

Data analysis

GIS

Service Oriented Architecture

Networks Communication Real time feedback (Digital Oilfield)

52 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

MDM in PPDM 3.8

53 Copyright 2013, PPDM Association. All Rights Reserved MASTER DATA MANAGEMENT

• The Well tables are being widely adopted within MDM

solutions. WELL

• This can be of tremendous benefit to industry as WELL_XREF WELL_ALIAS WELL_VERSION operators demand that companies that they work with adhere to the same standard

54 Copyright 2013, PPDM Association. All Rights Reserved MAPPINGS AND MAPPING RULES

Persist mappings in PPDM • Not in a spreadsheet Mappings PPDM_SYSTEM_MAP • Database to database or schema PPDM_MAP_DETAIL • Schema to database or schema

• Rule driven PPDM_MAP_RULE • PPDM mappings will be released in the sample data

PPDM PPDM 3.7.1 3.8

55 Copyright 2013, PPDM Association. All Rights Reserved MAPPING LEVELS

PPDM_SYSTEM_MAP PPDM_MAP_DETAIL

Which systems are you mapping?

• system to system • to table • to column • schema to column • information to be created (not mapped) • order to handle

56 Copyright 2013, PPDM Association. All Rights Reserved MAPPING RULES

PPDM_MAP_RULE • How is this mapping connected with mappings to other columns? • If a value is created, how is it created? • What are the min and max values that are acceptable? • If the condition is expressed procedurally, where is the code that validates? • If the condition is dependent on the value of another column, which one. • How are dates formatted? • What order to I process the rules in? • What version of the rule is this? • Is this the preferred rule? • What rule did I used last time I did a conversion? 57 Copyright 2013, PPDM Association. All Rights Reserved REFERENCE VALUES

Support for sophisticated reference PPDM_SYSTEM behaviour • Multiple sources • Hierarchy and granularity CLASS_LEVEL_COMPONENT PPDM_TABLE • Equivalences

• Cross Referencing PPDM_CODE_VERSION PPDM_COLUMN Sandbox to prepare reference values for use PPDM_CODE_VERSION_COLUMN

PPDM_CODE_VERSION_USE

PPDM_CODE_VERSION_XREF

PPDM_GROUP

R_CODE_VERSION_XREF

58 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

Meta Data in PPDM 3.8

59 Copyright 2013, PPDM Association. All Rights Reserved META MODEL – TABLE GROUPINGS

• PPDM_SYSTEM, PPDM_TABLE, PPDM_COLUMN

• PPDM_GROUP, PPDM_GROUP_OBJECT

• PPDM_CONSTRAINT, PPDM_CONS_COLUMN

• PPDM_SYSTEM_MAP, PPDM_MAP_DETAIL, PPDM_MAP_RULE

• PPDM_RULE, PPDM_RULE_DETAIL

• PPDM_EXCEPTION

• This will be detailed in Meta Model course

60 Copyright 2013, PPDM Association. All Rights Reserved

PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

Data Quality You cannot inspect quality into your data!

You can’t prove data is right. You can only prove it is not wrong.

61 Copyright 2013, PPDM Association. All Rights Reserved EFFECTIVE DATA MANAGEMENT STRATEGIES

Data Value

Data Master Data Governance Data Mgmt Quality

Data Data Data Organization Architecture Security

People Process

Technology

Effective Data Management

62 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY

Data Quality • Data Quality is fundamental to improving business workflows and decision-making processes • Billions of $ are lost annually in terms of inefficiency, poor decisions, and missed opportunities • Data quality initiatives are attracting a great deal of attention across all industries and within oil and gas in particular • However, quality programs can be overwhelming in scope, very expensive, and hampered by a scarcity of qualified resources • It is vital to focus available resources on addressing those data problems that have the most potential impact on the business through an analysis of data value

63 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY • Using our Data Value approach each data element is provided with a general value ranking on a scale of 1 to 4 • Level 1: Critical • Level 2: Important • Level 3: Useful • Level 4: Supportive • These value rankings can then be represented within the PPDM data model. • One approach to this would be to use the PPDM_GROUP tables to store value • As an example, the GROUP_NAME (in PPDM_GROUP) could be ‘Critical’ • Then each attribute falling in this category would be defined by TABLE_NAME and COLUMN_NAME in PPDM_GROUP_OBJECT

PPDM_GROUP

PPDM_GROUP_OWNER PPDM_GROUP_OBJECT

PPDM_GROUP_XREF PPDM_GROUP_REMARK 64 Copyright 2013, PPDM Association. All Rights Reserved GROUPS

Link objects in PPDM together to PPDM CODE support business processes VERSION In PPDM 3.8 Groups are used to associate tables into modules (52 PPDM TABLE subject areas) PPDM COLUMN • Core tables • Referenced tables PPDM GROUP • Supports the on-line documentation, which is divided by module PPDM PPDM RULE PROCEDURE • Supports the modular DDL Generator (on hold) PPDM PROPERTY

65 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY

• Using the same scale, value rankings could then be established for the different dimensions of Data Quality: • Accuracy • Currency • Timeliness • Consistency • Completeness • Standards • Again, the PPDM_GROUP tables could be used to store the value for each quality dimension for the different attributes • In this case, GROUP_NAME could be ‘Critical’ and GROUP_TYPE could be set to ‘Accuracy’ as an example. • Then each attribute falling in this category would be defined by TABLE_NAME and COLUMN_NAME in the PPDM_GROUP_OBJECT_TABLE

66 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY

67 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY

• Data Quality Rules for capturing and maintaining the attributes are then developed from the value rankings • Value I (Critical) data: Accuracy: Well location data should be validated from at least 2 independent sources Completeness: 100% of attributes should be populated Timeliness: New bid rounds should be entered within 5 days of announcement • Value II (Important) data: Completeness: 95% of attributes should be populated • …….

68 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY

PPDM_RULE PPDM_PROCEDURE

SYSTEM_ID SYSTEM_ID RULE_ID PROCEDURE_ID ACTIVE_IND ACTIVE_IND COLUMN_NAME EFFECTIVE_DATE EFFECTIVE_DATE EXPIRY_DATE PPDM_RULE EXPIRY_DATE PPDM_GUID PPDM_GUID PROCEDURE_DESC REMARK PROCEDURE_NAME RULE_CLASS PROCEDURE_TYPE RULE_DESC REMARK RULE_QUERY SOURCE P R_P P _FK RULE_PURPOSE TABLE_NAME SOURCE ROW_CHANGED_BY PPDM RULE allows you to keep track of TABLE_NAME ROW_CHANGED_DATE USE_CONDITION_DESC ROW_CREATED_BY USE_CONDITION_TYPE ROW_CREATED_DATE the data management rules that you apply ROW_CHANGED_BY ROW_QUALITY ROW_CHANGED_DATE ROW_CREATED_BY ROW_CREATED_DATE to creating, managing and using your ROW_QUALITY data. A good example of a business rule PROCEDURE_ID would be “If you populate a well location’s

latitude, you should also populate a

PRX_PR_FK

PRX_PR_FK2

PR_R_PRC_FK PRD_PR_FK

longitude value and the coordinate PR_R_PRP_FK PPDM_RULE_XREF

R_PPDM_RULE_CLASS PPDM PROCEDURE can PRG_PR_FK system reference”. SYSTEM_ID RULE_ID PPDM_RULE_DETAIL be used to store the R_PPDM_RULE_PURPOSE SYSTEM_ID2 RULE_ID2 XREF_OBS_NO SYSTEM_ID procedure or a pointer to a ACTIVE_IND RULE_ID PPDM_RULE_GROUP DESCRIPTION These rules can be expressed in plain DETAIL_SEQ_NO EFFECTIVE_DATE ACTIVE_IND procedure (or trigger, or EXPIRY_DATE AVERAGE_VALUE GROUP_ID PPDM_GUID narrative language or in structured form, AVERAGE_VALUE_OUOM SYSTEM_ID REMARK AVERAGE_VALUE_UOM RULE_XREF_TYPE script, or function, or RULE_ID BOOLEAN_RULE ACTIVE_IND SOURCE as procedures or SQL. PRE_PR_FK BUSINESS_RULE EFFECTIVE_DATE ROW_CHANGED_BY EFFECTIVE_DATE EXPIRY_DATE ROW_CHANGED_DATE application etc) that is used EXPIRY_DATE PPDM_GUID ROW_CREATED_BY MAX_DATE REMARK ROW_CREATED_DATE MAX_VALUE SOURCE ROW_QUALITY to enforce the rule. MAX_VALUE_OUOM ROW_CHANGED_BY MAX_VALUE_UOM ROW_CHANGED_DATE MIN_DATE ROW_CREATED_BY MIN_VALUE ROW_CREATED_DATE MIN_VALUE_OUOM ROW_QUALITY MIN_VALUE_UOM PPDM_GUID REFERENCE_SYSTEM_ID

REFERENCE_TABLE_NAME PRX_R_PRXT_FK REFERENCE_TABLE_NAME2 PPDM_RULE_ENFORCEMENT R_PPDM_RULE_XREF_TYPE REFERENCE_COLUMN_NAME REFERENCE_COLUMN_NAME2 REFERENCE_VALUE PPDM_PROCEDURE REFERENCE_VALUE_OUOM

PRG_PG_FK REFERENCE_VALUE_TYPE REFERENCE_VALUE_UOM PPDM_RULE_ENFORCEMENT REMARK PPDM_GROUP RULE_DETAIL_TYPE SOURCE PPDM RULE ENFORCEMENT SYSTEM_ID RULE_DESC RULE_ID GROUP_ID ROW_CHANGED_BY ENFORCEMENT_ID ACTIVE_IND ROW_CHANGED_DATE describes how the rule is ABBREVIATION ROW_CREATED_BY DEFAULT_UNIT_SYSTEM_ID ACTIVE_IND EFFECTIVE_DATE ROW_CREATED_DATE APPLICATION_ID EXPIRY_DATE ROW_QUALITY EFFECTIVE_DATE enforced, such as manually GROUP_NAME ENFORCE_METHOD GROUP_TYPE EXPIRY_DATE PPDM_GUID LONG_NAME during a load, or by insert trigger REMARK OWNER_BA_ID SOURCE PPDM_GUID ROW_CHANGED_BY PROCEDURE_ID or by monthly QC run etc. ROW_CHANGED_DATE REMARK R_PPDM_ENFORCE_METHOD ROW_CREATED_BY PRE_R_PR EM_FK SHORT_NAME ROW_CREATED_DATE SOURCE You can also define the owner of ROW_QUALITY ROW_CHANGED_BY ROW_CHANGED_DATE ROW_CREATED_BY this enforcement method, or the ROW_CREATED_DATE application or procedure used to ROW_QUALITY enforce the process.

69 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY: METRICS

70 Copyright 2013, PPDM Association. All Rights Reserved METRICS

PPDM_METRIC Data Loading Progress • Measured benchmarks PPDM_METRIC_COMPONENT

• Database loading and QC PPDM_METRIC_VALUE • Link to objects in PPDM

3000 Jan Feb 2500 Mar Apr 2000

1500

1000

500

0 Not Done Prepared Loaded QC Initial QC Verify

71 Copyright 2013, PPDM Association. All Rights Reserved QUALITY CONTROL AND AUDIT HISTORIES

How good is my data, and what have I done to validate it? Quality control by table, column, • Status history, process… Audit history • Column and row level (full audit-ability) • Table level (usually deletes)

PPDM_TABLE_HISTORY

PPDM_AUDIT_HISTORY PPDM_AUDIT_HISTORY_REM

PPDM_QUALITY_CONTROL

72 Copyright 2013, PPDM Association. All Rights Reserved QUALITY CONTROL

Who checked this value? When was it checked? How many checks have been completed? If the value has changed – what is the current QC status for the current value? What is the quality of this value? • How trustworthy is it? If the value is NULL, why is it NULL and is that OK? Who authorized the change? Who actually did it? How long should I keep this information?

73 Copyright 2013, PPDM Association. All Rights Reserved AUDIT HISTORIES

What was the original value? • Specific column for each data type What is the new value? • Specific column for each data type Where did the new information come from? Why was the information changed? How long should I keep this information’

74 Copyright 2013, PPDM Association. All Rights Reserved DATA QUALITY: WORKFLOW SUMMARY

Business Workflows

Decision Points Business Data Requirements Analysis

Data Value

Data Quality

Data Business Rules Analysis (PPDM) Metrics

Fix/Audit

75 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION Data Organization and Architecture

76 Copyright 2013, PPDM Association. All Rights Reserved EFFECTIVE DATA MANAGEMENT STRATEGIES

Data Value

Data Master Data Governance Data Mgmt Quality

Data Data Data Organization Architecture Security

People Process

Technology

Effective Data Management

77 Copyright 2013, PPDM Association. All Rights Reserved DATA ORGANIZATION

• Data organization refers to how data is structured within a company and the meaning assigned to that data • What something means to a geologist may be different to what it means to a reservoir engineer or, more so, to the financial people • PPDM has performed a lot of work to establish standard meanings to key data types and to define standard taxonomies • The ‘What is a Well?’ project has established a baseline for the definition of well components

78 Copyright 2013, PPDM Association. All Rights Reserved DATA ORGANIZATION

Ontologies Meta Data Taxonomies Physical

KID

Work Data Management Rules Policies, Practices & Procedures

79 Copyright 2013, PPDM Association. All Rights Reserved DATA ARCHITECTURE

Data Architecture • Data Architecture refers to the people, processes, and technology required to gather, store, maintain, and distribute data

Accum/ • Data must be Acquisition managed effectively Archival/ Deletion/ Storage across its full Renewal lifecycle to meet the

requirements Transform- Usage of the business ation

80 Copyright 2013, PPDM Association. All Rights Reserved DATA ARCHITECTURE

Data is Integrated Often PPDM Warehouse standards

based GIS GIS Systems Data is Managed As an Asset Combination System of PPDM standards SystemSystem of Record based and RecordRecord proprietary Data is Master Data Updated or Deleted

Often Source Source proprietary SystemSource SystemSource systems, with SystemSource Data is SourceSystem vendor System Created System developed data stores

81 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

Summary

82 Copyright 2013, PPDM Association. All Rights Reserved WHAT WE TALKED ABOUT

• Data management is a fundamental part of business • It’s not about IT • Understand the business (what it really is) • Understand how processes and SW damage data • Master data management adds value • Takes thought to plan and execute • It’s very hard to do, but it’s worth it • Master data management strategies and components • MDM without Data Governance may not add value • You can’t prove your data is right • But you can make a decision about whether to trust it

83 Copyright 2013, PPDM Association. All Rights Reserved PROFESSIONAL PETROLEUM DATA MANAGEMENT ASSOCIATION

Questions

84 Copyright 2013, PPDM Association. All Rights Reserved