Investigating a Constraint-Based Approach to Data Quality in Information Systems
Total Page：16
File Type：pdf, Size：1020Kb
Research Collection Master Thesis Investigating a Constraint-Based Approach to Data Quality in Information Systems Author(s): Probst, Oliver Publication Date: 2013 Permanent Link: https://doi.org/10.3929/ethz-a-009980065 Rights / License: In Copyright - Non-Commercial Use Permitted This page was generated automatically upon download from the ETH Zurich Research Collection. For more information please consult the Terms of use. ETH Library Investigating a Constraint-Based Approach to Data Quality in Information Systems Master Thesis Oliver Probst <[email protected]> Prof. Dr. Moira C. Norrie David Weber Global Information Systems Group Institute of Information Systems Department of Computer Science 8th October 2013 Copyright © 2013 Global Information Systems Group. ABSTRACT Constraints are tightly coupled with data validation and data quality. The author investigates in this master thesis to what extent constraints can be used to build a data quality manage- ment framework that can be used by an application developer to control the data quality of an information system. The conceptual background regarding the definition of data qual- ity and its multidimensional concept followed by a constraint type overview is presented in detail. Moreover, the results of a broad research for existing concepts and technical solu- tions regarding constraint definition and data validation with a strong focus on the JavaTM programming language environment are explained. Based on these insights, we introduce a data quality management framework implementation based on the JavaTM Specification Request (JSR) 349 (Bean Validation 1.1) using a single constraint model which avoids in- consistencies and redundancy within the constraint specification and validation process. This data quality management framework contains advanced constraints like an association con- straint which restricts the cardinalities between entities in a dynamic way and the concept of temporal constrains which allows that a constraint must only hold at a certain point in time. Furthermore, a time-triggered validation component implementation which allows the scheduling of validation jobs is described. The concept of hard and soft constraints is ex- plained in detail and supplemented with a implementation suggestion using Bean Validation. Moreover, we explain how constraints could be used to increase the data quality. A demon- strator application shows the utilisation of the data quality management framework. iii iv ACKNOWLEDGMENTS I am indebted to my supervisor David Weber who supported me during my master thesis and his colleague Dr. Karl Presser who gave us great insights into his point of view with respect to this master thesis topic. Thanks to global information systems group under the direction of Prof. Dr. Moira C. Norrie. Lastly, I would like to thank my colleagues, friends and family who have helped me in my work in any way. v vi CONTENTS I INTRODUCTION1 1 MOTIVATION3 2 STRUCTURE7 II CONCEPT BACKGROUND9 3 DATA QUALITY 11 4 DATA QUALITY DIMENSIONS 15 4.1 DATA QUALITY DIMENSION: DISCOVERY............. 15 4.1.1 THEORETICAL APPROACH.................. 15 4.1.2 EMPIRICAL AND INTUITIVE APPROACH......... 16 4.2 DATA QUALITY DIMENSION: DESCRIPTION............ 17 4.2.1 ACCURACY............................ 17 4.2.2 COMPLETENESS......................... 18 4.2.3 CONSISTENCY.......................... 19 4.2.4 TEMPORAL DATA QUALITY DIMENSIONS......... 20 4.2.4.1 TIMELINESS...................... 20 4.2.4.2 CURRENCY....................... 21 4.2.4.3 VOLATILITY...................... 21 4.2.5 OTHER DATA QUALITY DIMENSIONS........... 21 5 CONSTRAINT TYPES 23 5.1 DEFINITION................................ 23 5.2 TYPES................................... 24 5.2.1 DATA RULES........................... 24 5.2.2 ACTIVITY RULES........................ 27 III RESEARCH BACKGROUND 28 6 CROSS-TIER VALIDATION 29 6.1 CONSTRAINT SUPPORT IN MDA TOOLS: A SURVEY...... 29 6.2 INTERCEPTOR BASED CONSTRAINT VIOLATION DETECTION 30 vii viii CONTENTS 6.3 TOPES: REUSABLE ABSTRACTIONS FOR VALIDATING DATA. 31 7 PRESENTATION TIER VALIDATION 33 7.1 POWERFORMS.............................. 33 8 LOGIC TIER VALIDATION 35 8.1 CROSS-LAYER VALIDATION...................... 35 8.1.1 INTEGRATION OF DATA VALIDATION AND USER IN- TERFACE CONCERNS IN A DSL FOR WEB APPLICATIONS 35 8.2 PRESENTATION LAYER VALIDATION................ 37 8.2.1 MODEL-DRIVEN WEB FORM VALIDATION WITH UML AND OCL............................. 37 8.3 BUSINESS LAYER VALIDATION.................... 38 8.3.1 OVERVIEW AND EVALUTION OF CONSTRAINT VALID- ATION APPROACHES IN JAVA................ 39 8.3.2 LIMES: AN ASPECT-ORIENTED CONSTRAINT CHECK- ING LANGUAGE......................... 39 8.3.3 VALIDATION APPROACHES USING OCL.......... 41 8.4 DATA ACCESS LAYER VALIDATION................. 41 8.4.1 CONSTRAINT-BASED DATA QUALITY MANAGEMENT FRAMEWORK FOR OBJECT DATABASES......... 42 9 DATA TIER VALIDATION 43 9.1 OBJECT-ORIENTED DATABASES................... 43 9.2 RELATIONAL DATABASES....................... 43 IV TECHNOLOGY BACKGROUND 45 10 CROSS-TIER VALIDATION 47 10.1 BEAN VALIDATION........................... 47 10.1.1 JSR 349: BEAN VALIDATION 1.1............... 47 10.1.1.1 HIBERNATE VALIDATOR.............. 49 10.1.2 JSR 303: BEAN VALIDATION 1.0............... 50 10.1.2.1 APACHE BVAL..................... 51 11 PRESENTATION TIER VALIDATION 55 11.1 HTML5................................... 55 12 LOGIC TIER VALIDATION 59 12.1 CROSS-LAYER VALIDATION...................... 59 12.2 PRESENTATION LAYER VALIDATION................ 59 12.2.1 JSR 314: JAVASERVER™ FACES................ 59 12.2.1.1 ORACLE MOJARRA JAVASERVER FACES.... 61 12.2.1.2 APACHE MYFACES CORE.............. 61 12.2.1.3 APACHE MYFACES CORE AND HIBERNATE VALIDATOR...................... 64 CONTENTS ix 12.2.1.4 JSF COMPONENT FRAMEWORKS......... 66 12.2.2 GOOGLE WEB TOOLKIT.................... 68 12.2.3 JAVA™ FOUNDATION CLASSES: SWING........... 74 12.2.3.1 JFC SWING: ACTION LISTENER APPROACH.. 74 12.2.3.2 SWING FORM BUILDER............... 75 12.2.4 THE STANDARD WIDGET TOOLKIT............ 78 12.2.4.1 JFACE STANDARD VALIDATION.......... 79 12.2.4.2 JFACE BEAN VALIDATION............. 82 12.2.5 JAVAFX.............................. 85 12.2.5.1 FXFORM2........................ 85 12.3 BUSINESS LAYER VALIDATION.................... 87 12.4 DATA ACCESS LAYER VALIDATION................. 88 12.4.1 JSR 338: JAVA™ PERSISTENCE API............. 88 12.4.1.1 ECLIPSELINK..................... 91 12.4.1.2 HIBERNATE ORM................... 93 12.4.1.3 DATANUCLEUS.................... 95 12.4.2 JSR 317: JAVA™ PERSISTENCE API............. 97 12.4.2.1 APACHE OPENJPA.................. 98 12.4.2.2 BATOO JPA....................... 100 12.4.3 NON-STANDARD JPA PROVIDERS.............. 101 12.4.3.1 HIBERNATE OGM................... 101 12.4.3.2 VERSANT JPA..................... 105 12.4.3.3 OBJECTDB....................... 106 12.4.3.4 KUNDERA........................ 107 13 DATA TIER VALIDATION 113 14 TECHNOLOGY OVERVIEW 115 V APPROACH 120 15 DATA QUALITY MANAGEMENT FRAMEWORK 121 15.1 BASIS.................................... 121 15.2 FEATURES................................. 123 15.3 PERSISTENCE.............................. 126 15.4 CONSTRAINTS AND DATA QUALITY DIMENSIONS....... 128 15.5 DEMO APPLICATION.......................... 128 16 ASSOCIATION CONSTRAINT 131 16.1 STATIC ASSOCIATION CONSTRAINT................ 132 16.1.1 SIMPLE @Size METHOD.................... 132 16.1.2 SUBCLASSING METHOD.................... 134 16.2 DYNAMIC ASSOCIATION CONSTRAINT............... 136 16.2.1 TYPE-LEVEL CONSTRAINT METHODS........... 137 16.2.1.1 HAND-CRAFTED ASSOCIATION CONSTRAINT METHOD........................ 137 x CONTENTS 16.2.1.2 GENERIC ASSOCIATION CONSTRAINT METHOD 139 16.2.1.3 INTROSPECTIVE ASSOCIATION CONSTRAINT METHOD........................ 143 16.2.2 ASSOCIATION COLLECTION METHOD........... 146 17 TEMPORAL CONSTRAINT 151 17.1 DATA STRUCTURE........................... 151 17.1.1 TEMPORAL INTERFACE.................... 151 17.1.2 PRIMITIVE TEMPORAL DATA TYPES............ 152 17.1.3 TEMPORAL ASSOCIATION COLLECTION......... 153 17.1.4 DATA STRUCTURE EXTENSION............... 153 17.2 CONSTRAINTS.............................. 154 17.2.1 @Deadline CONSTRAINT................... 154 17.2.1.1 ANNOTATION..................... 154 17.2.1.2 VALIDATOR...................... 155 17.2.2 CONSTRAINT COMPOSITION................. 156 17.2.2.1 @AssertFalseOnDeadline CONSTRAINT.... 157 17.2.2.2 @MinOnDeadline CONSTRAINT.......... 158 17.2.3 TEMPORAL CONSTRAINT CREATION........... 159 18 TIME-TRIGGERED VALIDATION COMPONENT 161 18.1 TTVC: SCHEDULERS.......................... 161 18.2 TTVC: JOBS AND JOBDETAILS.................... 163 18.2.1 TTVC: BASIC JOB........................ 163 18.2.2 TTVC: ABSTRACT VALIDATION JOB............ 163 18.2.3 TTVC: ABSTRACT JPA VALIDATION JOB......... 164 18.2.4 TTVC: UNIVERSAL JPA VALIDATION JOB......... 166 18.3 TTVC: TRIGGERS............................ 166 18.4 TTVC: JOB LISTENER......................... 167 18.5 TTVC: PERSISTENT VALIDATION REPORT............ 168 19 HARD AND SOFT CONSTRAINTS 173 19.1 DEFINITION................................ 173 19.1.1 HARD CONSTRAINT...................... 173 19.1.2 SOFT CONSTRAINT....................... 175 19.1.3 SUMMARY............................. 176 19.2 IMPLEMENTATION........................... 176 19.2.1 HARD CONSTRAINT IMPLEMENTATION.......... 177 19.2.2 SOFT CONSTRAINT IMPLEMENTATION.......... 177 19.2.2.1 PAYLOAD-TRY-CATCH METHOD......... 177 19.2.2.2 SOFT CONSTRAINTS VALIDATOR......... 178