Wcs-Overview.Pdf

OGC Coverages: Data & Services

Peter Baumann Jacobs University | rasdaman GmbH

. Motivation: What is a “coverage”? . Coverage data: the OGC Coverage Model . Coverage services: the WCS Suite . Conformance Testing . Implementations . Summary

sensor feeds [OGC SWE]

coverage server

sensor feeds [OGC SWE]

coverage server

. WCS: one generic schema for all coverage types; n-D; scalable; versatile processing → access & processing services

SOS WCS

data images data data

feature coverage meta FE WCPS CQL

WFS-T WCS-T CS-T

WFS WMS WCS CS-W

• WMS "portrays spatial data pictures" • WCS: "provides data + descriptions; data with original semantics, may be interpreted, extrapolated, etc.“ [09-110r3]

. Motivation: What is a “coverage”? . Coverage data: the OGC Coverage Model . Coverage services: the WCS Suite . Conformance Testing . Implementations . Summary

Coverages Tutorial :: © 2013 Peter Baumann7 OGC Coverages . Coverage = "space-time varying phenomenon“ - ISO 19123 (=OGC Abstract Topic 6) - Today typically raster, but more defined (curved grids, TINs, meshes, ...) - historically constrained to x/y ...hm, x/y/t...hm, x/y/z/t...hm...pressure? . Web Coverage Service (WCS) = coverage access service - Get original data (or subset thereof), suitable for further processing - www.ogcnetwork.net/wcs . Coverage-related working groups within OGC: - WCS.SWG - Coverages.DWG Coverages Tutorial :: © 2013 Peter Baumann8 Coverage Definition ISO 19123 class GML 3.2.1 Application Schema for Coverages is abstract → many different «FeatureType» GML::Feature implementations possible contains hook → not per se for metadata interoperable

«FeatureType» OGC coverage std Coverage is concrete and interoperable

domainSet rangeType rangeSet

«Union» «type» «Union» GML::DomainSet SWE Common::DataRecord GML::RangeSet

«FeatureType» Abstract Coverage Grid Coverage

MultiSolid Coverage Rectified GridCoverage MultiSurface Coverage MultiCurve Coverage Referenceable MultiPoint GridCoverage Coverage

. Not georeferenced, „just pixels“ - GMLCOV::GridCoverage . Georeferenced, regular, possibly oblique - GMLCOV::RectifiedGridCoverage . 1+ irregular axes - All axes irregular: GML 3.3 ReferenceableGridByVectors * - GMLCOV::ReferenceableGridCoverage . 1+ axes warped - All axes warped: GML 3.3 ReferenceableGridByArray * - GMLCOV::ReferenceableGridCoverage

. Pure GML: complete coverage represented by GML . Special Format: other suitable file format (ex: MIME type “image/tiff”) . Multipart-Mixed: multipart MIME, type “multipart/mixed”

GML Coverage GML Coverage NetCDF GeoTIFF Domain set Domain set Domain set Domain set Range type Range type Range type Range type Range set xlink Range set Range set App Metadata App Metadata App Metadata App Metadata NetCDF file

. Coverage has slot „metadata“ allowing to link in kind of metadata - WCS will deliver this, even without knowing contents . Ex: EO-WCS GetCoverage result . Inverse possible, too: metadata record contains (or links) coverage

. Motivation: What is a “coverage”? . Coverage data: the OGC Coverage Model . Coverage services: the WCS Suite . Conformance Testing . Implementations . Summary

WCS offering can be seen as a single virtual document

Notwithstanding other actual encodings

GML coverage

Hook for future service- related coverage metadata

. GetCapabilities - what service extensions? - What formats? CRSs? - What coverages?

. DescribeCoverage - coverage metadata

. GetCoverage - coverage, or subset thereof

. Purpose: Get information about - Coverage offering: identifiers of coverages available & (possibly) coverage types - Service capabilities: which formats? which CRSs? can this server do reprojection? etc.

. Request: http://www.acme.com/wcs?SERVICE=WCS&VERSION=2.0.1 . Response: “Capabilities document”: ... http://www.opengis.net/spec/WCS_protocol-binding_soap/1.0/data/conf-class/soap http://www.opengis.net/spec/WCS_coverage-encoding_netcdf/1.0/data/conf-class/netcdf http://www.opengis.net/spec/WCS_service-model_processing/1.1/data/conf-class/processing ...

. Get whole or subset of coverage . Format encoding selected - in coverage‟s Native CRS by MIME type identifier • No information loss - default: coverage„s Native Format - subset = trim | slice • No information loss - Supported formats advertised in Capabilities doc

http://www.acme.com/wcs? SERVICE=WCS & VERSION=2.0 & REQUEST=GetCoverage & COVERAGEID=C1 & SUBSET=Lat(0,100) & SUBSET=Long(10,50) & FORMAT=image/tiff more examples: http://standards.rasdaman.org Coverages Tutorial :: © 2013 Peter Baumann19 Sample Satellite Image Timeseries WCS

«FeatureType» Coverage . OGC Coverages: time just another axis domainSet rangeType rangeSet «Union» «type» «Union» DomainSet DataRecord RangeSet . WaterML: timseries = time slices - WaterML extended: scalars → images - Inefficient time subsetting

GML SWE Common OWS Common

GML Application Schema - WCS Core Coverages

Protocol Format Encoding Data Model Service Binding Usability

GeoTIFF Quality WCS-T GET-KVP Multilinguality

netCDF Hierarchies Processing POST-XML

JPEG2000 Range SOAP Subsetting

GMLJP2 Scaling REST

GRIB2 JSON CRS

. Select components from compound cells - „channel“, „band“ subsetting - Not to be mixed up with (domain) subsetting) . By way of example: ...& RANGESUBSET=red &… …& RANGESUBSET=nir,red,green &… …& RANGESUBSET=green,red,blue &… …& RANGESUBSET=nir:green &… …& RANGESUBSET=band01,band03:band05,band19:band21 &…

. In adoption process

. Retrieve coverage in CRSs different from Native CRS - WCS Core access always in coverage„s Native CRS → result, as per Core, always delivers exact, unaltered cell („pixel“) values . By way of example: ... & OUTPUTCRS=http://www.opengis.net/def/crs/EPSG/0/4326 & ...

. CRS Name Type Specification [OGC 11-135] defines CRS URL framework - All CRSs in OGC expressed as URLs - Ex: http://www.opengis.net/def/crs/EPSG/0/4326 - Horizontal space, vertical space, time, „abstract“ axes, array indexing - Not part of WCS

. Client can enforce interpolation behavior whenever server needs to interpolate - Scaling, reprojection - If this extension is not supported: server silently chooses . By way of example: ... & INTERPOLATION=http://www.opengis.net/def/interpolation/cubic &… …& INTERPOLATION= lat:http://www.opengis.net/def/interpolation/linear, long:http://www.opengis.net/def/interpolation/linear, time:http://www.opengis.net/def/interpolation/nearest-neighbor &…

. In adoption process

“XQuery for rasters”: ad-hoc navigation, extraction, aggregation, analytics

. Time series

. Image processing

. Summary data

. Sensor fusion & pattern mining

. "From MODIS scenes M1, M2, and M3, the absolute of the difference between red and nir, in HDF-EOS"

for $c in ( M1, M2, M3 ) return (hdf , encode( A hdf , abs( $c.red - $c.nir ), B "hdf“ hdfC) )

. "From MODIS scenes M1, M2, and M3, the absolute of the difference between red and nir, in HDF-EOS" - …but only those where nir exceeds 127 somewhere

for $c in ( M1, M2, M3 ) where some( $c.nir > 127 ) return (hdfA, encode( hdfC) abs( $c.red - $c.nir ), "hdf“ )

. "From MODIS scenes M1, M2, and M3, the absolute of the difference between red and nir, in HDF-EOS" - …but only those where nir exceeds 127 somewhere - …inside region R

for $c in ( M1, M2, M3 ), $r in ( R ) where some( $c.nir > 127 and $r ) return (hdfA) encode( abs( $c.red - $c.nir ), "hdf“ )

. merge WCPS with XQuery FLWOR let $r := doc(“WCPS”) //coverage/metadata/region/[ @name = "Austria" ] for $c in doc(“WCPS”) //coverage/[ some( $c.nir > 127 and $r ) ] ) return encode( abs( $c.red - $c.nir ), "image/tiff" )

for $c in doc(“WCPS”)//coverage/[ some( $c.nir > $c.red )] return { $c/@id } { $c/boundedBy } . Implementation: Jacobs University & Athena Research Lab federation of eXist + rasdaman

. Motivation: What is a “coverage”? . Coverage data: the OGC Coverage Model . Coverage services: the WCS Suite . Conformance Testing . Implementations . Summary

. OGC TEAM Engine („Test, Evaluation, and Measurement“) for conformance testing - XML-controlled scripts - Technically, test = set of XML specifications . Following modular specification and testing policy (OGC 08-131), - standards consist of requirements classes = indivisble units of implementation - Requirements classes consist of requirements = individual statements on functionality - For each requirement there is a corresponding conformance test in the Abstract Test Suite (ATS) – typically: Annex 1 of specification document - For each ATS there is an Executable Test Script (ETS) – ideally; under work

. Motivation: What is a “coverage”? . Coverage data: the OGC Coverage Model . Coverage services: the WCS Suite . Conformance Testing . Implementations . Summary

. High acceptance by implementers for WCS 2 . Users increasingly planning to move to WCS 2 (eg, NGA) . Known implementations: - rasdaman, MapServer, GeoServer, OPeNDAP, GMU, Trellis, ... - WCS Core Reference implementation : rasdaman • Array DBMS, fully multi-dimensional; arrays in database or file archives • WMS, WCS, WCS-T, WCPS, WPS • 

Coverages Tutorial :: © 2013 Peter Baumann WCS Core, WCPS Reference Implementation . rasdaman („raster data manager“) www.rasdaman.org - Array („raster“) DBMS for massive n-D raster data - database or pre-existing archive . Flexible querying: SQL with nD image processing select img.green[x0:x1,y0:y1] > 130 from LandsatArchive as img

. Scalable parallel “tile streaming” architecture

array A

select max((A.nir - A.red) / (A.nir +A.red)) from A

select max((A.nir - A.red) / (A.nir + A.red)) Array B - max((B.nir - B.red) / (B.nir + B.red)) from A, B select max((B.nir - B.red) / (B.nir + B.red)) from B

. Scalable On-Demand Processing for the Earth Sciences - 6 databases with 100+ TB for all Earth sciences + planetary science . Platform: rasdaman - Distributed query processing, integrated data/metadata search, 3D clients - Strictly open standards

. Motivation: What is a “coverage”? . Coverage data: the OGC Coverage Model . Coverage services: the WCS Suite . Conformance Testing . Implementations . Summary

Coverages Tutorial :: © 2013 Peter Baumann42 WCS Evolution . WCS 1.0.0 . Model extension - GML harmonization & unifying, service - 34p + 15p annexes independent coverage model - First attempt; limited to lat/lon + time; - Beyond raster: curvi•linear grids, in places inconcise, no rigorous general meshes, … testing - N-D coverages . WCS 1.1.0 - Increased domain support - 56p + 60p annexes . Engineering aspects - More concise, but complex (65p CRS discussion!) - Separate data from service model - Corrigenda: WCS 1.1.1, 1.1.2 - Concise semantics - Improved testability . WCS 2.0 - Core/extension modularization - All coverage types, nD, harmonized, - Crisp & easy to implement testable, interoperable - Allow for efficient & scalable implementations

. WCS -- simple coverage access (subsetting, transforms, ...) . WCPS -- on-demand processing & filtering by raster query language . WPS -- on-demand processing & filtering by server code . SOS -- sensor data acquisition

coverages1 WCS ... coverages coverages WCS-T coverage 2 ... WCPS ... offerings coverages sensor data SOS (any format) 3 WPS ...

coverages4 Coverages Tutorial :: © 2013 Peter Baumann Coverages FAQ . Can„t I serve coverages through a WFS? - Yes; a coverage is a feature, so a WFS can deliver coverages, too - but coverages induce bespoke extra functionality, such as subsetting, scaling, ... . Can„t I serve coverages through a Web Map Service? - Coverages can be served via WMS, but appear as pictures (think of elevation!) . Coverages are specified in GML, but I want other formats - Coverage model (+WCS) based on GML for uniform, format-independent semantics - other representations (ie: encodings) possible, specification under work for GeoTIFF, NetCDF, etc. . How do WPS and WCPS relate? - WCPS bridges WCS and WPS in a harmonized, interoperable manner: WCS for simple access; WCPS for query processing; WPS for free algorithm coding - Under work: WPS Application Profile for Coverages Processing = WCPS

. I am using Sensor Web Enablement, why do I need WCS? - Coverages are a fundamental data structure, SWE is application oriented - SWE standards make use of coverages (eg, O&M) • best practice in OGC: O&M for metadata, coverages for „payload“ data - Coverage definition harmonized with SWE by using SWE Common for range type - Separation of coverage data and service model → coverages can be exchanged between different OGC services, such as: • SOS → WCS-T → WCS or WCPS or WPS • ...just as is the case with features and SWE / WFS - Summary: the application-oriented SWE standards use the domain-neutral coverage standards, thereby achieving interoperability and cross-standard harmonization on coverage level

. n-D coverages main contributor to Big Geo Data = Sensor, image, simulation, & statistics data - High volumes, complex analytics . WCS Core, WCPS Reference Implementation: rasdaman

. See also: - www.ogcnetwork.net/wcs - standards.rasdaman.org

Coverages Tutorial :: © 2013 Peter Baumann Document Overview . Coverage data structure: - 09-146r2 GML Application Schema for Coverages - 12-100 GeoTIFF Coverage Encoding Extension - 12-108 JPEG2000 Coverage Encoding Extension - 11-010 NetCDF Encoding Format Extension . Coverage service: - 09-110r4 WCS 2.0 Core - 12-101 WCS - GeoTIFF Coverage Encoding Extension - 09-147r1 WCS KVP protocol extension - 09-148 WCS XML/POST protocol extension - 09-149 WCS XML/SOAP protocol extension - 08-059 WCS - ProcessCoverages Extension - 12-040 WCS – Range Subsetting Extension - 12-039 WCS - Scaling Extension - 11-053 WCS - CRS Extension - 12-049 WCS - Interpolation Extension . Other: - 08-068r2 Web Coverage Processing Service (WCPS) Language - 10-140 WCS 2.0 - Earth Observation Application Profile - 11-135 Name Type Specification for CRSs external.opengeospatial.org/twiki_public/CoveragesDWG/WebHome Coverages Tutorial :: © 2013 Peter Baumann48