Project acronym e-SOTER Project full title Regional pilot platform as EU contribution to a Global Soil Observing System Project No 211578 e-SOTER validation and accuracy assessment Deliverable D10 March 2012 SEVENTH FRAMEWORK PROGRAMME Environment ENV.2007.4.1.3.3 Development of a Global Soil Observing System Report Deliverable No D10 e-SOTER Document Information Deliverable number D10 Deliverable title e-SOTER validation and accuracy assessment Period covered n.a. Due date of deliverable 29/02/2012 Actual date of deliverable Author(s) Bas Kempen, Gerard Heuvelink (Alterra), Tereza Zádorová, Vít Penížek (CULS), Jacqueline Hannam (CU), Rainer Baritz, Ulrich Schuler (BGR) Participants All WP4 partners Work package 4 Work package title Accuracy assessment of terrain and soil platforms Dissemination level PU Version 1.0 History of Versions Version Date Status Author Approval level 1.0 06/03/2012 Draft Bas Kempen 2.0 14/03/2012 Final Bas Kempen 2 Report Deliverable No D10 e-SOTER Table of Contents Table of Contents .................................................................................................................................... 3 1. Executive Summary ............................................................................................................................. 4 2. Introduction ........................................................................................................................................ 6 3. Landform Validation ........................................................................................................................... 8 3.1 Methodology .............................................................................................................................. 8 3.2 Application to Western European window .............................................................................. 11 3.3 Application to CE window ......................................................................................................... 16 4. Soil Validation ................................................................................................................................... 21 4.1 Methodology ............................................................................................................................ 21 4.2 Application to UK part of the Western European window ....................................................... 24 4.3 Application to German-Czech part of the Central European window ...................................... 34 5. Uncertainty Propagation Analysis ..................................................................................................... 46 5.1 Methodology ............................................................................................................................ 46 5.2 Application to the Western European pilot area ..................................................................... 49 5.3 Application to Central European pilot area .............................................................................. 55 6. Validation of the WP3 landform map ............................................................................................... 62 6.1 Methodology ............................................................................................................................ 62 6.2 Results ...................................................................................................................................... 63 7. Conclusions and recommendations .................................................................................................. 70 8. References ........................................................................................................................................ 73 Appendix 1. Site, soil profile and topsoil data of the NSI dataset (UK) ................................................ 74 Appendix 2. Landform classification levels of the WP3 maps .............................................................. 77 3 Report Deliverable No D10 e-SOTER 1. Executive Summary This deliverable reports on the validation and uncertainty propagation analysis of e-SOTER products by comparing the e-SOTER soil and landform maps with independent validation data and by analysing how errors in a Digital Elevation Model (DEM) propagate to the e-SOTER landform map. Landform validation was done by comparing the true landform in the Western European (WE) and Central European (CE) windows with the landform as depicted on the e-SOTER maps. The various simplification and generalization steps of the e-SOTER landform classification methodology cause discrepancies between the true and predicted landform. Validation showed that the accuracy of the e-SOTER landform maps is high for landform classes ‘elevation’, ‘relief intensity’ and ‘flatness’, with agreement between true and predicted class in 81-98% of cases. For landform class ‘slope’ the agreement is only around 50%, which means that in 1 out of 2 cases the e-SOTER map does not agree with reality. This is caused by the highly fragmented spatial pattern of the true slope map, a feature that cannot be reproduced in the e-SOTER map because it must generalize the map to the 1:1000,000 million scale. Comparison of landform validation results between the WE and CE windows did not show large or meaningful differences. Soil validation was based on a comparison of the dominant soil classes on the e-SOTER maps with independent soil observations derived from existing legacy data. This first required a conversion of soil classes from the local classification system to the World Reference Base (WRB). For this purpose correlation tables were prepared. The validation results show that the e-SOTER soil map of the UK pilot area reproduced the large scale patterns and had an agreement with the ‘true’ soil class of 51%. The e-SOTER map overrepresented Histosols and Podzols and lacked Leptosols as a dominant soil group. Validation results in the German / Czech pilot area revealed an overall agreement of only 32%. The rather low purity in this case can be assigned to high variability of the soil cover and often low dominancy of the dominant soil unit in the SOTER units. Also, in this case the more detailed validation data allow the use of more strict validation criteria than in the UK case. Overall, the fairly low agreements between e-SOTER soil maps and validation data in both pilot areas show that the soil maps have large uncertainties, even at the coarse soil group level that was considered here. Part of the disagreement may be caused by errors in the validation data, but this is unlikely to be the major cause of the discrepancy. The uncertainty propagation analysis showed that DEM uncertainty mainly affects the e-SOTER slope class. The real elevation is more noisy than the smoothed DEM and this causes the DEM-derived slope to be too small. Elevation was hardly affected while flatness is only sensitive to DEM errors in relatively flat areas and relief intensity in areas with a more intense relief (i.e. the CE pilot area). Overall DEM uncertainty does not seriously impair landform accuracy and is mainly restricted to zones along class boundaries. It is a smaller source of uncertainty than the simplification and generalization steps that were analysed in the landform validation procedure. 4 Report Deliverable No D10 e-SOTER Validation of the WP3 landform maps for the UK area of the Western European window in terms of predictability of the WRB reference soil groups indicated that the hillshed analysis gives the best overall results. Both the hillshed and the object-oriented approach give better results than the WP1 map at subclass level, although differences in predictability and purity are modest. The entropy is equal for all three landform maps, indicating that the WP3 landform units are not internally more homogeneous with respect to soil distribution than the WP1 units. 5 Report Deliverable No D10 e-SOTER 2. Introduction The e-SOTER project used much of its resources to automate and improve the SOTER methodology such that it yields reproducible results that utilise new sources of (remote sensing) information that have recently become available. The new methodology and results for pilot areas and windows have been described in great detail in project deliverables D3, D5 and D8. However, the new methodology and resulting products only have merit when these are sufficiently accurate for the intended use. It is therefore important to validate the resulting maps and analyse how errors in the inputs to the e-SOTER algorithms propagate to the output. This deliverable tackles these issues by confronting the e-SOTER maps of landform and soil with independent validation data and by analysing how errors in the Digital Elevation Model (DEM) propagate through e-SOTER landform classification algorithms. The term ‘validation’ is defined here as “the process of determining the degree to which a product is an accurate representation of the real world”. Thus landform validation requires a comparison of ‘true’ landform with landform produced with the e-SOTER methodology, and likewise soil validation makes a comparison of ‘true’ soil classes with soil classes predicted by the e-SOTER methodology. In order to derive the true landform it is first necessary to define it. In the e-SOTER project the landform at some location is completely determined by the altitude at the location and its surroundings. This implies that landform validation data can be obtained from