Mapserver, Rasdaman, and Eoxserver) Enabling the Online Data Access to Heterogeneous Multi-Dimensional Satellite Data

786

Introducing WCS 2.0, EO-WCS, and Open Source Implementations (MapServer, rasdaman, and EOxServer) Enabling the Online Data Access to Heterogeneous Multi-dimensional Satellite Data

Christian SCHILLER, Stephan MEISSL, Peter BAUMANN, Stephan KRAUSE, Gerhard TRIEBNIG, Fabian SCHINDLER-STRAUSS, Andrei AIORDACHIOIAE, Jinsongdi YU und Dimitar MISEV

Abstract

While the Open Geospatial Consortium (OGC) Web Map Service (WMS) is widely known and deployed as View Service to serve portrayals of data (e.g. maps as images), the access to true data (Download Service) via the OGC Web Coverage Service (WCS) has, so far, rarely been implemented. Special needs exist for Earth Observation (EO) satellite Datasets. The recently published Version 2.0 of WCS, if combined with the new WCS Earth Observation Application Profile (EO-WCS), provides the required functionalities for subsetting in area and time of interest, reprojecting, accessing selectable (spectral) bands, extracting of Time-Series and Stitched Mosaics, and the integration into service chains. Furthermore, standardized EO metadata are exchanged with each EO-WCS service request. This paper informs about the new features of WCS 2.0 (OGC 09-110r3) and the new EO-WCS (OGC 10-140). Furthermore, the Open Source reference implementations of WCS 2.0 (within MapServer 6.0, and rasdaman) and of EO-WCS (EOxServer) are presented.

Introduction

The European Space Agency (ESA) is coordinating the Heterogeneous Mission Accessibility (HMA) initiative (HMA WIKI, 2011) to foster international standardization relating to the information infrastructure for Earth Observation (EO). In this context ESA actively cooperates with the Open Geospatial Consortium (OGC) like through the Online Data Access project (HMA-FO Task-3, ESA contract No. 22507/09/I-LG, industrial team EOX, Jacobs University Bremen, Spot Image, GIM), which has supported the specification of WCS 2.0 (OGC 09-110r3) and the EO–WCS (OGC 10-140) and some implementations.

OGC Web Coverage Service 2.0

The OGC Web Coverage Service (WCS) supports electronic retrieval of geospatial data as “coverages” (information representing space/time-varying phenomena) for client-side exploitation. WCS provides detailed descriptions, defines a rich syntax for requests, and returns data in its original semantics (as opposed to the pictures provided by WMS).

Strobl, J., Blaschke, T. & Griesebner, G. (Hrsg.) (2011): Angewandte Geoinformatik 2011. © Herbert Wichmann Verlag, VDE VERLAG GMBH, Berlin/Offenbach. ISBN 978-3-87907-508-9. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/). Introducing WCS 2.0, EO-WCS, and Open Source Implementations 787

The definition of WCS 2.0 (OGC 09-110r3) is based on OGC's Core/Extension Model, the GML Application Schema for Coverages (OGC 09-146r1), OWS Common (OGC 06- 121r9), and OGC Abstract Topic 6 (OGC 07-011). It supports coverage types given in OGC 09-146r1. Following OGC 09-110r3 the operations required by a WCS implementation are: (a) GetCapabilities – service metadata and information on offered coverages. Example: http://wcs-server/ows?service=WCS&version=2.0.0&request=GetCapabilities (b) DescribeCoverage – detailed metadata on selected coverages metadata, Example: http://wcs-server/ows?service=wcs&version=2.0.0&request=DescribeCoverage& coverageid=Test_Scene_1 (c) GetCoverage – request for processing and returning of a selected coverage including domain subsetting operation which is subdivided into trimming and slicing. A trim is performed on a dimension and a lower and upper bound. A slice is performed on a dimension and a position. Examples: http://wcs-server/ows?service=wcs&version=2.0.0&request=GetCoverage& CoverageId=Test_Scene_1&FORMAT=image/tiff&mediatype=multipart/mixed &subset=x(2100,2300)&subset=y(3870,4070) http://wcs-server/ows?service=wcs&version=2.0.0&request=GetCoverage& CoverageId=Test_Scene_1&FORMAT=image/tiff& SUBSET=x,http://www.opengis.net/def/crs/EPSG/0/4326(17,17.4)& SUBSET=y,http://www.opengis.net/def/crs/EPSG/0/4326(48,48.3)& OutputCRS=http://www.opengis.net/def/crs/EPSG/0/3035& &SIZE=x(200)&SIZE=y(200) OGC's Core/Extension Model foresees that the Core of new standards shall only stipulate basic functionalities and definitions. For WCS 2.0 this means that upcoming Extensions to the WCS Core, or Application Profiles, may add additional operation types (see EO-WCS). For a WCS implementation the Core plus at least one protocol binding (e.g. GET-KVP, POST-XML, or SOAP), and one Coverage encoding format Extension (e.g. GeoTIFF, NetCDF, or JPEG2000) are required. Extensions may reference existing definitions of earlier versions (e.g. WCS 1.1, OGC 07-067r5).

WCS 2.0 Earth Observation Application Profile (EO-WCS)

The EO-WCS specializes the generic WCS 2.0 by bundling several specifications and adding requirements on an implementation. Commonly used EO domain terminology is mapped into the EO-WCS terminology as given in Table 1. 788 C. Schiller, S. Meißl, P. Baumann et al.

Tab. 1: Vocabulary as used in the EO-WCS (not comprehensive)

EO terminology EO-WCS terminology scene / image / product / raster data Dataset (image) / raw data (image) /orthoimage mosaic / seamless mosaic Dataset google-like mosaic / composite Stitched Mosaic mosaic (not radiometrically adjusted) series / Time-Series Dataset Series collection Dataset Series rectified grid grid having regular spacing in a projected or geographic CRS referenceable grid not rectified, but associated with (one or more) coordi- nate transformations which relate the image or engi- neering CRS to a projected or geographic CRS

To efficiently handle and manage Time-Series, Stitched Mosaic Datasets, and related EO metadata additional concepts (Figs. 2, 3, and 4) are introduced in EO-WCS. It defines EO coverages which have a latitude/longitude or projected x/y spatial extend and a temporal validity extend. They are derived from rectified and referencable coverage (Fig. 1a and 1b) as defined in OGC 09-146r1. Further, a Dataset Series (Fig. 1c) is defined as a collection (virtual container) of Stitched Mosaics and/or Datasets where the Dataset Series itself is not a coverage. Since the DescribeCoverage request targets only coverages, the need for the specification of the new operation request DescribeEOCoverageSet, to gather information about a Dataset Series and its content, becomes evident. This DescribeEOCoverageSet request allows to limit the query by applying trimming and slicing in space and time.

dataset series a) b) c)

Fig. 1: EO-WCS concepts: rectified / referencable grid coverage and Dataset Series

The coverageIDs provided in the DescribeEOCoverageSet result enables the user (client) to issue repeated GetCoverage requests to access the actual data of the Dataset Series. Using this approach of repeated single requests fosters interoperability, reduces the burden on the supplier side (better load balancing) and does not require the definition and use of 3D- CRSs. The second new concept introduced in EO-WCS is that of Stitched Mosaics (Fig. 2). These are homogeneous collections of spatially non-overlapping subsets of Datasets, accessible as coverages. Introducing WCS 2.0, EO-WCS, and Open Source Implementations 789

Fig. 2: EO-WCS concept of a Stitched Mosaic

This means Datasets are patched together (google like), while sharing a common grid. One example might be “Provide a mosaic of all available data for the current growing season, with the newest Dataset on top”. A Stitched Mosaic can be interpreted (i.e., requested) as a single coverage where the Datasets have to keep their respective EO metadata over the full lifetime. This implies that when new images arrive they will replace older parts and that the existing EO metadata for these pixels have to be adjusted accordingly, as has the footprint of the underlying image part. The Dataset Series concept introduced in EO-WCS permits that Datasets are members in multiple Dataset Series. This enables satellite data providers, which often have huge archives organized in collections (e.g. based on sensors), to make those collections available via an EO-WCS interface without the need for data duplication (Fig. 3). Every EO Coverage contains EO Metadata, consisting of an EarthObservation record as defined in the OGC Earth Observation Metadata Profile of Observations and Measurements (OGC 10-157r2) and a lineage describing the history of operations leading to the coverage on hand. More details can be found in OGC 10-140.

Fig. 3: Various Datasets (e.g. a collection) may again form together a Dataset Series

Finally, the GetCoverage request provides the access mechanism to the data. The GetCoverage request is unchanged over WCS Core (OGC 09-110r3), except that for EO coverages slicing is disallowed, as it would leave the EO Metadata undefined. 790 C. Schiller, S. Meißl, P. Baumann et al.

Open Source Implementations

MapServer “MapServer”1 is probably the most famous Open Source web mapping project. Its strength lies in the fast and efficient serving of raster data, which was one of the main reasons why it has been chosen for the first implementation of the new WCS 2.0. This included the support of XML/POST protocol binding, which is now also available to WCS 1.1. These and many other improvements are included in the upcoming release of MapServer 6.0. The implementation is limited to grid coverages (Fig. 1a,b). and can be seen as the first Reference Implementation which proves the overall concepts. rasdaman “rasdaman”2is the second Open Source implementation which provides access via a WCS 2.0 interface. It is a middleware which extends standard relational database systems with the ability to store and retrieve multi-dimensional raster data (arrays) of unlimited size through an SQL-style query language (rasql) allowing to phrase select/insert/update/delete statements against a raster database. Queries are optimized on server side. The petascope component (a servlet package) of rasdaman provides service interfaces based on the OGC WCS and also Web Coverage Processing Service WCPS (Fig. 4a).

Fig. 4: (a) WCPS Reference Implementation architecture with rasdaman; (b) Logical architecture of EOxServer

EOxServer “EOxServer”3 is a new, but highly active, Open Source project and aims to provide a comprehensive server side solution, primarily for satellite data providers, to offer an easy online access mechanism to their extensive data holdings for further large scale analysis of time-series and to provide up-to-date large scale mosaics. It implements EO-WCS and targets the handling and management of offered coverages and their associated EO

1 MapServer – http://www.mapserver.org 2 rasdaman – http://rasdaman.eecs.jacobs-university.de/trac/rasdaman or http://www.rasdaman.com 3 EOxServer – http://www.eoxserver.org Introducing WCS 2.0, EO-WCS, and Open Source Implementations 791 metadata. It builds on top of the trusted workforce MapServer-GDAL/OGR-PostgreSQL/ PostGIS (Fig. 4b)) to perform coverage manipulation and serving. EOxServer is build using Python/Django and provides an administration web-interface to manage the offered Datasets, Dataset Series, and Stitched Mosaics. They are ingested and managed together with associated EO metadata. The full content of multiple directories may be ingested at once. The application of filter expressions (e.g. 1999*.TIF) allows a fine grained selection process. EOxServer also provides scripting possibilities to automate tasks e.g. to hourly synchronize data holding of an rolling FTP archive for offering these via an EO-WCS interface.

Summary

The new OGC WCS 2.0 and EO-WCS have been defined and demonstrated in practice to fulfill the requirements of an EO Download Service for meta-data handling, preview, subsetting, reprojection, resizing, mosaicking, etc. and for the creation of interoperable, user controlled or fully automated end-to-end web service chains. An overview of the new specifications, and the first available Open Source tools (MapServer 6.0, rasdaman, EOxServer) implementing them have been presented.

References

HMA WIKI EOPORTAL, JOIN AND SHARE (o. J.): http://wiki.services.eoportal.org/ tiki-index.php?page=HMA%20Wiki (15.03.2011). OGC 04-107 (2004): The OpenGIS Abstract Specification Topic 7: The Earth Imagery Case. ver.5.0, 27-09-2004 (identical with ISO 19101-2 Working Draft #3). OGC 06-121r9 (2010): Web Services Common Standard, ver. 2.0.0, 2010-04-07. OGC 07-011 (2007): Abstract Specification Topic 6: The Coverage Type and its Subtypes, ver. 7.0 (identical to ISO 19123:2005), 2007-12-28. OGC 07-067r5 (2008): Web Coverage Service (WCS) Implementation Standard ver. 1.1.2, Version 1.1 Corrigendum 2 release, 2008-03-19. OGC 09-110r3 (2010): WCS Interface Standard - Core, ver. 2.0.0, 2010-10-27. OGC 09-146r1 (2010): GML Application Schema - Coverages, ver. 1.0.0, 2010-10-27. OGC 09-147r1 (2010): Web Coverage Service 2.0 Interface Standard - KVP Protocol Binding Extension, ver. 1.0.0, 2010-10-27. OGC 09-148r1 (2010): Web Coverage Service 2.0 Interface Standard - XML/POST Protocol Binding Extension, ver. 1.0.0, 2010-10-27. OGC 09-149r1 (2010): Web Coverage Service 2.0 Interface Standard - XML/SOAP Protocol Binding Extension, ver. 1.0.0, 2010-10-27. OGC 10-140, (2011): Web Coverage Service 2.0 Interface Standard – Earth Observation Application Profile, ver. 0.4.0, 2011-04-16 . OGC 10-157r2 (2010): Earth Observation Metadata profile of Observations and Measurements, ver. 0.2.0, 2010-11-11.