DATA EXCHANGE IN GEOTECHNICAL ENGINEERING by Nazila Mokarram A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (CIVIL ENGINEERING) December 2010 Copyright 2010 Nazila Mokarram Dedication to my parents Nourintaj Pashaeirad, Ali Mokarram my sister Nazanin Mokarram and my husband Soheil Seiqali ii Acknowledgments I would like to express my gratitude to everyone who has helped me with this study. Particularly, my sincere gratitude goes to my advisor, Prof. Jean-Pierre Bardet, for his guidance and support during the entire course of my doctoral program. Special thanks are extended to my committee members, Prof. Geoffrey R. Martin, Prof. John Wilson, and Prof. L. Carter Wellford, for their constructive criticism and advices. Special appreciation also goes to Dr. Behnam Hushmand, for his encouragement and support. I would like to recognize the special contribution and many helpful suggestions from my colleagues in the geotechnical engineering group Mr. Amir Zand and Dr. Fang Liu, who generously shared with me their knowledge and experiences. My gratitude also extends to my other colleagues Dr. Jinaping Hu, Dr. Rana Alfares, and Dr. Yangsoo Lee for building a friendly atmosphere at University of Southern California. Finally, I appreciate my family’s continuing understanding and support, especially, my best friend and husband, Soheil Seiqali, who has always been present to help me with my biggest challenges. iii Table of Contents Dedication ii Acknowledgments iii List of Tables vii List of Figures ix Abstract xiv Chapter 1. Introduction 1 1.1. Exchange of Geotechnical Information 1 1.2. Research Objectives 2 1.3. Organization of Dissertation 2 Chapter 2. Exchange of Geotechnical Information 5 2.1. Geotechnical Community and Information 6 2.1.1. Geotechnical Community 6 2.1.2. Geotechnical Data 8 2.2. Data Exchange in Geotechnical Community 12 2.3. Scope of This Study 16 2.4. Evolution of Geotechnical Information Release 19 2.4.1. AGS and AGSML 20 2.4.2. NGES 20 2.4.3. NEES 21 2.4.4. GeotechML 22 2.4.5. DIGGS 22 2.4.6. GML-Conformant Spatial Modeling 23 2.5. Review of AGS Format 23 2.5.1. Data Dictionary and Base Data 24 2.5.2. Data Groups and Unique Identifiers 26 2.5.3. File Format 28 2.5.4. Two Table Tests 29 2.5.5. Unit Types 32 2.5.6. Data Integrity and Transfer 34 2.6. Analysis of DIGGS Format 36 2.7. Requirements for Geotechnical Data Exchange Format 40 2.8. Summary 50 iv Chapter 3. From Relational to XML-Based Data Organization 52 3.1. AGS Data Structure Analysis 52 3.1.1. AGS Conceptual Modeling 52 3.1.2. AGS Data Group Types 56 3.1.3. Relational Characteristics of AGS 58 3.2. Transformation of Relational Information to XML Format 63 3.2.1. Extensible Markup Languages (XML) 63 3.2.2. Conversion Algorithms 64 3.2.3. Principles of Transformation 66 3.3. XML-Based Data Organization 66 3.3.1. Preliminary Model: Implementation of Basic Mapping Rules 66 3.3.2. XAGS 1.0: Exploring Nested Structure 69 3.3.3. XAGS 2.0: Building Relationships with Identifiers 73 3.4. XAGS 2.0 Implementation 73 3.4.1. Schema Symbols 75 3.4.2. Type, Simple Type and Complex Type 76 3.4.3. Project 77 3.4.4. Borehole 79 3.4.5. In Situ Tests 79 3.4.6. Sample 87 3.4.7. Laboratory Tests 89 3.5. XAGS 2.0 Deliberations 91 3.5.1. Unique Identifiers 91 3.5.2. File Element 92 3.5.3. Coordinate System 93 3.5.4. Data Structure Options for Serialized Data 94 3.6. Summary 99 Chapter 4. XAGS 2.0 Data Usage, Validation, Exchange and Distribution 102 4.1. Data Generation and Modification 103 4.1.1. XML editors 103 4.1.2. Random Data Generation 107 4.1.3. Spreadsheets for Data Generation and Modification 107 4.2. Display of Borehole Data 109 4.2.1. Extensible Style sheet Language Family (XSL) 109 4.2.2. Non-spatial Data Display with HTML 111 4.2.3. Spatial Data Display with Interactive Maps 113 4.3. Structural Consistency and Data Validation 116 4.3.1. Unit Enumeration 117 4.3.2. Missing Data Recognition and Unit Validation 117 4.3.3. Data Format and Range Validation 118 4.3.4. Unique Relationships 119 4.4. Data Exchange between Teams and Team Members 120 4.4.1. Exchange of Complete Data Set 120 v 4.4.2. Exchange of Data Subsets 121 4.5. XAGS and Other Formats 126 4.5.1. Conversion of XAGS Data to Other Formats 126 4.5.2. Conversion of Existing Data Sets to XAGS Format 128 4.6. Archive and Distribution via World Wide Web 131 4.6.1. System Architecture 131 4.6.2. User Interface and Major Functions 133 4.7. XAGS Evaluation 137 4.8. Summary 143 Chapter 5. Future Direction: Metadata Models 145 5.1. Background 146 5.2. Metadata Modeling History for Collaborative Research 146 5.3. Background on Metadata Modeling 148 5.4. The metadata model 150 5.4.1. Premises of metadata model 150 5.4.2. Classes 153 5.4.3. Summary of relationships in metadata model 154 5.5. Applications of metadata model 156 5.5.1. Naming objects for building relationships 156 5.5.2. Metadata entry 157 5.5.3. Documentation of data sets 158 5.5.4. Exchange of data sets documented with XML metadata 159 5.5.5. Automatic generation of data reports 160 5.6. Discussion 161 5.7. Summary 162 Chapter 6. Summary and Conclusions 164 6.1. Summary of Research 164 6.2. Major Contributions 166 6.3. Possible Future Considerations for XAGS 168 References 171 Appendices Appendix A: XAGS 2.0 Schema Files 179 Appendix B: XAGS 2.0 Sample XSL File 197 vi List of Tables Table 2.1 A sample matrix of practice-oriented geotechnical entities involved in a site investigation project and their most probable data usage types 11 Table 2.2 AGS compatible geotechnical software programs, their categories and acceptable operating systems (GGSD, http://www.ggsd.com/) 25 Table 2.3 Geotechnical laboratory tests defined by AGS data dictionary, their number of attributes and their corresponding British Standard and ASTM standard tests 27 Table 2.4 AGS data dictionary for direct shear test in two tables: Shear Box Testing General (SHBG) and Shear Box testing (SHBT) (AGS 2004) 31 Table 2.5 AGS data dictionary abbreviation pick list for standard units (AGS 2004) 32 Table 2.6 List of requirements for a geotechnical data exchange format 48 Table 3.1 The main data groups in AGS, their definitions and immediate higher level data group 57 Table 3.2 Possible combinations of immediate upper level data groups and data types 58 Table 3.3 AGS data groups combination types based on Table 3.2 combinations 59 Table 3.4 AGS geotechnical laboratory tests and the classification of identifiers into foreign and regular keys and their parent groups 61 Table 3.5 Symbolic representation of XML schema diagrams used in this dissertation (after XMLSpy) 76 Table 4.1 A conceptual example of different hierarchy access levels of team members in a geotechnical projects 125 vii Table 4.2 Main Tables defined in Access DB 129 Table 4.3 Comparison of the data formats in satisfying data exchange format requirements for geotechnical projects 142 Table 5.1 The 16 classes of the metadata model 153 Table 5.2 List of all relations between objects of the metadata model 154 Table 5.3 Naming objects from their attributes for building relations 157 viii List of Figures Figure 2.1 Geotechnical community members involved in compiling a borehole log (Borehole log image from Bardet et al., 1999) 7 Figure 2.2 Different types of geotechnical data produced and transformed within a sample geotechnical project. Not all data types can be easily exchanged. 9 Figure 2.3 Possible problem in data comprehension, without personal communications or prior knowledge between team members by lack of uniform geotechnical exchange format 13 Figure 2.4 Data flow diagrams (DFD) of current state and future usage of data in a sample geotechnical engineering project. a) A sample data flow diagram of geotechnical engineering projects. b) Second level DFD for current state of extracting input data from test results. c) Second level DFD for future vision of extracting input data from test results. 15 Figure 2.5 Two different graphical presentations of CPT data. Left: graphical presentations of CPT results in three graphs, right: graphical presentations of CPT results in one combined graphs (Bardet et al., 1999). 17 Figure 2.6 Two different graphical presentations of SPT data. Left: graphical presentations of SPT results, right: Sample boring log, showing stratigraphy, physical sampling record and SPT blow counts (ROSRINE, 2008) 19 Figure 2.7 Relationships between Project, Hole, Sample and Specimen in AGS 28 Figure 2.8 AGS two level data structure for direct shear test including summary information in one table (SHBG) and detailed data in another (SHBT) 30 ix Figure 2.9 A part of AGS data file, showing PROJ, HOLE, SAMP, CLSS, SHBG, SHBT and FILE data groups. The highlighted parts should all be carried for preserving referential integrity if only shear box test information wants to be transferred. 36 Figure 3.1 Partial EER diagram of AGS geotechnical laboratory tests including borehole (HOLE), sample (SAMP), classification (CLSS), shear tests (SHBG, SHBT) and file attachments (FILE).