DataZoo: an Oceanographic Information System Supporting Scientific Research Karen S. Baker, Mason Kortz, and James Conners Integrative Oceanography Department Scripps Institution of Oceanography, UCSD This report is about the development, management, and continuing design of an information system within a field-oriented, scientific research environment in collaboration with those who collect and use the data. Scripps Institution of Oceanography Technical Report June 2011 Table of Contents 1 Introduction ......................................................................................................................... 3 2 Background .......................................................................................................................... 5 2.1 Initial Circumstances ...............................................................................................................5 2.2 LTER Context ...........................................................................................................................7 2.2.1 Scientific culture ................................................................................................................................7 2.2.2 Information management ................................................................................................................8 2.2.3 Site-based network model...............................................................................................................9 2.3 Ocean Informatics Context.................................................................................................. 10 2.3.1 Information management philosophy....................................................................................... 11 2.3.2 Enacting the information management philosophy............................................................. 12 2.4 Drawing Upon Conceptual Foundations ...................................................................... 12 3 Context of Use....................................................................................................................13 3.1 Scientific Organization of Data........................................................................................... 14 3.2 Scientific Uses of Data........................................................................................................... 15 3.3 Impact of Local Considerations.......................................................................................... 15 3.4 Balancing Tensions ................................................................................................................ 16 4 Technical Architecture ....................................................................................................17 4.1 History of Development ........................................................................................................ 17 4.1.1 Information System Prior to DataZoo (Before 2005)......................................................... 18 4.1.2 First Generation (2005 - 2007)................................................................................................... 19 4.1.3 Second Generation (2007 - 2010).............................................................................................. 19 4.1.4 Third Generation (2010 - ongoing)........................................................................................... 20 4.2 Application Architecture...................................................................................................... 21 4.2.1 Conceptual Framework................................................................................................................. 21 4.2.2 Implementation................................................................................................................................ 24 4.3 Data system interface ............................................................................................................ 29 4.3.1 Data...................................................................................................................................................... 30 4.3.2 Management ..................................................................................................................................... 36 4.3.3 Resources........................................................................................................................................... 38 4.3.4 Summary............................................................................................................................................ 39 4.4 Influences and Impacts......................................................................................................... 40 4.4.1 Major Technologies ....................................................................................................................... 40 4.4.2 Design Philosophies....................................................................................................................... 41 4.4.3 Community Needs.......................................................................................................................... 42 5 Discussion ...........................................................................................................................43 5.1 Organizational Situation and Perspectives on Data....................................................... 43 5.2 Change and Flexibility.......................................................................................................... 45 5.3 Specialization and Simplification ....................................................................................... 46 5.4 Developing Policies and Publishing Data.......................................................................... 46 5.5 Developing our Practices...................................................................................................... 47 5.6 Considering Future Design Issues...................................................................................... 48 6 Final Thoughts .................................................................................................................50 7 References.........................................................................................................................52 1 8 Appendices........................................................................................................................59 8.1 DataZoo Timeline................................................................................................................. 59 8.2 DataZoo Flyer ........................................................................................................................ 62 8.3 Applications for Heterogeneous Data Types .............................................................. 63 8.3.1 Multi-component information system................................................................................. 63 8.3.2 Complex, Irregularly Sampled Measurement Data ........................................................ 65 8.3.3 Very Large: Filefinder ................................................................................................................. 70 8.4 Login Registration and Data Policies ............................................................................ 71 8.4.1 DataZoo Login................................................................................................................................. 71 8.4.2 ZooDB Login.................................................................................................................................... 72 8.4.3 EuphausiidDB Login .................................................................................................................... 72 8.4.4 IchthyoDB Login............................................................................................................................ 73 8.5 DataZoo ................................................................................................................................... 74 8.5.1 Schemas............................................................................................................................................. 74 8.5.2 Web Interface Views.................................................................................................................... 76 8.6 Data Portal ............................................................................................................................. 83 8.6.1 Palmer LTER ................................................................................................................................... 83 8.6.2 CCELTER ........................................................................................................................................... 84 8.7 Additional Elements............................................................................................................ 85 8.7.1 Publishing Data into DataZoo .................................................................................................. 85 8.7.2 Zooplankton Dataspace.............................................................................................................. 86 8.7.3 Ocean Institute ............................................................................................................................... 87 8.7.4 Site-Network Distributed Model ...........................................................................................