Corine Land Cover

Summary

Introduction 3

Source data transformation 5

‘Referenced’ vs ‘Embedded’ encoding 5

GML dataset Validation Results 7

Steps to the creation of the Deegree CLC WFS instance 8

File size issue for the SQL feature store 9

Possible solution to the file size issue 9

HALE WFS-T experimental function 9

Loading WFS-served CLC features in QGIS 10

WFS served CLC features : validation results 11

Steps to the creation of the Deegree CLC WMS instance 12

Loading WMS LC.LandCover layer in QGIS 13

Deegree Webservices endpoints 13

Issues related to the creation of the GeoServer LC Webservices 14

Conclusions 16

Introduction

This document describes how:

1. the CLC source data[1] already used in the EAGLE project [2] and available as sqlite file[3] was transformed into a gml dataset file compliant to INSPIRE Land Cover Vector application schema using HALE software.

The choice of using the same EAGLE demo source dataset (the first 10 records of each CLC-code for 3 pilot countries) has been made to compare results and encountered issues.

2. WFS and WMS services were set up in Deegree[4] environment serving the abovementioned Corine Land Cover geospatial dataset compliant to INSPIRE LCV application schema[5].

3. the issues encountered in GeoServer environment hindered the set-up of GeoServer WFS and WMS services for LC GML dataset.

It’s worth noting that:

· the mapping rules provided by EAGLE reference project were used for the transformation process.

Nevertheless, a different ‘encoding of the mapping’ was used with respect to the member relationship linking the Land Cover dataset to its Land Cover units.

As will be better explained in the following of the document, this change was needed to overcome issues related to correct access to and visualisation of Corine Land Cover GML features in GIS environment.[6]

· in the Land Cover harmonisation process, the huge amount of data to transform and serve is clearly a limitation both in the GML dataset production and in the web services set up.

To overcome the issue related to huge file size ingestion in Deegree SQL data stores, the use of HALE WFS-T experimental function was tested and the results are reported in the following of the document.

· to solve the “orientation of polygons not counterclockwise” – data validation issue related to the respect of the ISO19107[7] - the “Unify Winding order” new HALE feature has been tested and result reported.

Source data transformation

Source data were mapped into INSPIRE Land Cover Vector application schema structure

· according to the LC_Matching_Table_CLC_pub.xlsx mapping rules provided by EAGLE project

· making use of HALE transformation software.

As already stated in the introduction, a new encoding was used for the mapping of the member relationship (see Figure 1), linking the LandCoverDataset to its LandCoverUnits.

More specifically the ‘referenced encoding’ was used instead of the ‘embedded encoding’ (this latter was used in the EAGLE project).

Figure 1

‘Referenced’ vs ‘Embedded’ encoding

The LandcoverDataset can refer each of its LandCoverUnit by means of:

· an external link to the unit (@xlink href)

· a description embedded in the LandCoverDataset feature itself.

Figure 2 illustrates the target schema (i.e. INSPIRE LandCoverVector) structure as displayed in HALE workbench.

As can be seen, two choices are available to map the member attribute of the LandCoverDataset feature type:

1. reference choice (in the Figure2 is the mapping option highlighted in green in the red box) in which the LandCoverUnits are addressed by means of xlink:href . This way each LandCoverUnit is described/mapped as a different gml feature member.

2. embedded choice (the mapping option highlighted in light blue in the red box) in which the description of the LandCoverUnits is embedded in the LandCoverDataset feature i.e. not available as stand-alone feature type.

Figure 2

Despite the encodings being both formally correct, only the ‘referenced approach’ allows:

1. the correct visualisation of the INSPIRE GML dataset (imported as GML file or by means of WFS Getfeature operation) in the GIS environment

2. to download LandCoverUnits from WFS without having to download the entire dataset (this is not possible when using the embedded approach).

GML dataset Validation Results

Different tools were used to assess the conformity of the transformed GML dataset to

· INSPIRE Land Cover Vector target schema

· GML encoding.

It’s important to remark that HALE and Oxygen tools only perform a validation against those schema requirements that can be expressed by means of XML schema grammar (i.e. xsd). Conversely, the eENVplus Validation Service goes deeper into validation, since it performs also the validation of other GML Specs requirements (that cannot be expressed through xsds) by means of OGC GML schematron file[8] and OGC GML Test Suite Java methods (TestNG).

Note that the produced GML files were validated as well against the Land Cover Vector v4.0 application schema requirements included in the Land Cover schematron[9] developed by Epsilon Italia in the framework of the eENVplus project and modified by EAGLE project developers.

Following Table1 reports the validation results.

Validated GML File name / HALE
Validation / OXYGEN
Validation / eENVplus VS 2.0
Validation
(based on OGC CITE test suite rel.1.22) / Schematron Validation Oxygen + LandCoverVector_v3.4.sch schematron / Schematron Validation eENVplus VS + LandCoverVector_v3.4.sch schematron
clc_completeDataset / Passed / Out of memory / Out of memory / Out of memory / Out of memory
clc_1000 / Passed / Passed / schema validation test = passed
overall validation result= failed / Passed / Passed

Table 1

The eENVplus Validation Service validation fails because some Corine Land Cover polygons in the dataset (those corresponding to the Madeira island for example) do not fall into the area of the Coordinate Reference System addressed by the dataset.

Initially the validation issue “orientation of polygons not counterclockwise” was reported by the eENVplus Validation Service. This validation issue is related to the respect of the ISO19107 requiring orientation of exterior polygon boundary counter clockwise and interior boundaries clockwise.

This issue was solved later on thanks to the “Unify Winding order” new HALE feature.

In the exporting GML phase, this new feature allows to leave the exterior polygon boundary winding order ‘as-is’, or to unify to CW or CCW. For inner polygon boundaries, the reverse order -with respect to the exterior one- is used.

The clc_1000.gml dataset was re-exported selecting the “Unify Winding order to counter clockwise” and no issue related to orientation of polygons was reported.

Steps to the creation of the Deegree CLC WFS instance

Step 1:

A CLC workspace was created and activated in Degree.

An empty PostGis DB [10]was created to serve as a repository for the CLC data. This empty database is used by Deegree to create tables and views according to the INSPIRE LC application schema (see step 2 below).

A jdbc connection to CLC PostGis DB was established in Degree.

Step 2:

A CLC SQL feature store was set up selecting ‘schema-driven’ mode + ‘relational mapping’.

This way the detailed feature type definitions and property declarations from CLC application schema were used by Deegree to create relevant tables in the abovementioned PostGis DB (the Deegree wizard provides an ad-hoc configuration dialog where the user can enter all settings required to establish a connection to the PostGIS database).

Step 3:

The transformed INSPIRE Land Cover GML dataset[11] was used to populate the PostGIS tables by means of the feature store Loader function.

Step 4:

A WFS service was set up and a slight modification of the automatically produced configuration file was necessary:

1. to use urn encoding for GML 3.2 CRS info. This was necessary to avoid incorrect display of the served datasets in some GIS environment (for example in QGIS)[12].

2. to introduce the INSPIRE extended capabilities

File size issue for the SQL feature store

The main issue encountered in setting up WFS server for the CLC data is related to the limit size of the GML file to be ingested in the data store by means of the Loader function (Step 3). A ‘Java heap space’ issue prevented the ingestion of the complete CLC dataset (+255000 features, almost 1.5 GB size) while smaller size datasets ingestion (up to 10000 features) was successful.

Possible solution to the file size issue

To overcome the issue related to huge file size ingestion in Deegree SQL data stores, the use of HALE WFS-T experimental function was tested.

HALE WFS-T experimental function

HALE WFS-T feature allows to publish GML to a transactional Web Feature Service, for instance to publish INSPIRE data to a Deegree server. There are two different choices for publishing to WFS:

· Direct upload: performs a single request that starts as soon as the first transformed features are available.

· Partitioned upload:splits the transformed data before uploading and performs multiple requests to the WFS-T (recommended for large data sets). The partitioning is done based on the references between features.

We used the ‘WFS_T Partined Upload’ to publish CLC dataset to the Deegree WFS instance. We set ‘15000- instances per transaction’ partitions.

After about 24 hours, 60000 features were ingested in the database (i.e. 4 transactions correctly completed), but then the HALE running task got stuck (though no error message appeared in the HALE workbench nor could be found in HALE log) and we aborted execution.

In Deegree log we found following message

“java.io.IOException: com.ctc.wstx.exc.WstxIOException: java.io.IOException at org.deegree.services.controller.utils.HttpResponseBuffer.flushBuffer(HttpResponseBuffer.java:301)..”

We also tried re-running WFS_T Partined Upload task from HALE specifying 10000- instances per transaction partitions and got same java.io.IOException issue.

The issue is being reported to HALE developers (though we are not sure whether reported issues depend on HALE or if they are related to Deegree server configuration or memory management).

Loading WFS-served CLC features in QGIS

Using the WFS Client 2.0 QGIS plugin and specifying the server url:

http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/EEA_LC_WFS?

the LC.LandCover layer can correctly be imported, displayed and queried.

Moreover, it correctly overlays on the CLC spatiaLite source data layer (see Figure 3 below).

Figure 3

WFS served CLC features : validation results

It was found that, in creating the SQL datastore from INSPIRE LandCoverVector application schema, Deegree did not create the elements related to LandCoverNomenclature.xsd and to gmd.xsd both imported by the LandCoverVector.xsd.

This is because, when the element structures extend a certain complexity (with nesting and recursion) Deegree simply omits them in order to avoid the config files to blow up.[13]

Manual intervention was needed to modify the SQL datastore configuration schema automatically created by Deegree. More specifically needed elements in the LandCoverDataset feature type were mapped according to the following :

Steps to the creation of the Deegree CLC WMS instance

Manual configuration was needed for the creation CLC WMS according to following steps:

1. Creation of a CLC style store and styles contained therein

2. Creation of CLC layer store and layers contained therein

3. Creation of CLC theme (theme can be thought of as a collection of layers, organized in a tree structure)

The WMS CLC layer can be visualised at

http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/console/wms/wms.html

clicking on the ‘+ ‘ icon on the right and selecting LC.LandCover layer

Loading WMS LC.LandCover layer in QGIS

The WMS can be added to and correctly displayed in QGIS map.

Deegree Webservices endpoints

The WFS can be accessed at

http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/EEA_LC_WFS?

below are a WFS GetFeature examples for LCUnits and LCDataset respectively

http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/EEA_LC_WFS?service=WFS&version=2.0.0&request=GetFeature&count=100&typename=lcv:LandCoverUnit

http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/EEA_LC_WFS?service=WFS&version=2.0.0&request=GetFeature&count=100&typename=lcv:LandCoverDataset

The WMS can be accessed at

http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/services/EEA_WMS?

The LC.LandCover layer can be visualised at

http://cloud.epsilon-italia.it:8085/deegree-webservices-3.3.18/console/wms/wms.html

clicking on the ‘+ ‘ icon on the right and selecting relevant layer

Issues related to the creation of the GeoServer LC Webservices

The HALE 2.9.4 GeoServer App-Schema Integration feature was used to automatically convert the HALE alignment into an app-schema configuration for GeoServer and setup INSPIRE compliant services by means of the GeoServer App-Schema plugin.

Initially we tried the App-Schema Configuration [Direct Upload] (i.e. the configuration is generated and immediately uploaded to GeoServer via its REST API) but we got a “org.apache.http.client.HttpResponseException: Internal Server Error” message.

It’s worth specifying here that the GeoServer compatibility mode in HALE workbench was selected and reported no issue (green check sign).

We then used the App-Schema Configuration function and manually copied the produced files the GeoServer data directory.

The LandCoverVector store as well as the LandCoverDataset and LandCoverUnit layers were automatically created by GeoServer after the relevant app-schema file was uploaded.

The WFS serving LandCoverDataset is correctly functioning at

http://cloud.epsilon-italia.it:8083/geoserver/ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=lcv:LandCoverDataset

This WFS string successfully validates in the eENvplus Validation Service.

Conversely, the following error arises when invoking the WFS 2.0 for LandCoverUnit, for example with the following string:

http://cloud.epsilon-italia.it:8083/geoserver/ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=lcv:LandCoverUnit&count=10

ServiceException

java.lang.RuntimeException: Error applying mapping with targetAttribute lcv:geometry/gml:Surface/gml:polygonPatches Error applying mapping with targetAttribute lcv:geometry/gml:Surface/gml:polygonPatches Could not find working property accessor for attribute (the_geom) in object