FUSION OF OPTICAL AND SAR SATELLITE DATA FOR IMPROVED LAND COVER MAPPING IN AGRICULTURAL AREAS

T. Riedel, C. Thiel, C. Schmullius

Friedrich-Schiller-University Jena, Institute of Geography, Earth Observation, Loebdergraben 32, D-07743 Jena, Germany, Email: [email protected]

ABSTRACT pixel, feature or decision level. With the availability of multifrequency and high-resolution spaceborne SAR Focus of this paper is on the integration of optical and data such as provided by the TerraSAR-X and PALSAR SAR data for improved land cover mapping in ALOS mission an increased interest in tools exploiting agricultural areas. The test site is located east of the full information content of both data types will arise. Nordhausen, , Germany. From April to December 2005 Landsat-5 TM, ASAR APP and ERS-2 Objective of this paper is a comparison of different data were acquired continuously over the test site image fusion techniques for optical and SAR data in building up a comprehensive time series. Regarding the order to improve the classification accuracy in agricul- fusion of optical and SAR data following three aspects turally used areas. Furthermore, the potential of will be addressed by this paper. The value of different multitemporal SAR data for land cover and crop type methodologies for the synergistic usage of both data mapping will be demonstrated. In this context optimal types is subject of a first analysis. Multitemporal SAR image parameters for the derivation of basic land cover images provide an important data base for land cover classes will be defined. On base of these findings and crop type mapping issues. This will be (amongst other things) a processing chain for the auto- demonstrated in the second section of this paper. mated generation of basic land cover products will be Finally, a classification scheme for the generation of introduced. basic land cover maps using both optical and SAR data will be presented. With respect to operational 2. STUDY AREA AND EXPERIMENTAL DATA applications the proposed procedure should have a high potential for automation. The study area “Goldene Aue” is located east of Nord- hausen at the southern border of the middle mountain 1. INTRODUCTION range , North Thuringia, Germany and is charac- terized by intensive agricultural land use. Main crop The availability of up-to-date and reliable land cover types are winter wheat, rape, corn and winter barley and crop type information is of great importance for (Fig. 1). many earth science applications. For operational appli- cations the development of robust, transferable, semi- automated and automated approaches is of special inter- est. In regions with frequent cloud cover such as Central Europe the number of suitable optical data is often lim- ited. The all-weather capability is one major advantage of SAR data beyond optical systems. Furthermore, radar sensors provide information complementary to those contained in visible-infrared imagery. In the optical range of the electromagnetic spectrum the information depends on reflective and emissive characteristics of the Earth surface, whereas the radar backscatter coefficient is primarily determined by structural and dielectric at- tributes of the surface target. The benefit of combining optical and SAR data for improved land cover mapping was demonstrated in several studies [1, 2, 3]. Multisen- Figure 1. Location of the test site and crop type map sor image data analyses are often performed without an from 2005 alteration of the digital numbers amongst the different data types. The term image fusion itself is defined as From April to December 2005 optical and SAR data ‘the combination of two or more different images to were acquired continuously over the test site building up form a new image by using a certain algorithm’ [4]. In a comprehensive time series. During the main growing general, the data fusion process can be performed on season (April – late August) 2 Landsat-5 TM, 9 Envisat

______Proc. ‘Envisat Symposium 2007’, Montreux, Switzerland 23–27 April 2007 (ESA SP-636, July 2007)

ASAR APP and 6 ERS-2 scenes were recorded (Fig. 2). distance varies between 0 and 1414, whereas 0 signifies On July 10 optical and SAR data were acquired nearly no and 1414 a very high separability. simultaneously providing an excellent data base for image fusion analysis. Parallel to each satellite overpass Main objective of this study was to set up an automatic extensive field data were obtained including crop type, and transferable classification scheme for the derivation growth stage and vegetation height. of basic land cover categories. The proposed processing chain – shown in Figure 3 – is composed of three main stages. The first step comprises the segmentation of the optical EO-data using the multiresolution segmentation approach [6] implemented in the eCognition software. Next, for each land cover class potential training sites will be selected automatically on base of a decision tree. As the application of fixed thresholds sometimes will Figure. 2 EO-data base fail, the thresholds values for reflectance, backscattering coefficient, ratios and texture information specified in 3. METHODOLOGY the nodes of the decision tree will be adapted to each EO-scene separately. To achieve this, for each land All EO-data were pre-processed on base of widely used cover type an optimal set of characteristic image pa- standard techniques. As parts of the test site are charac- rameters was defined by analyzing the time series avail- terized by significant topography, the normalization able in a systematic manner. Additionally, information procedure introduced by Stussi et al. [5] was applied to reported in literature and libraries (e.g. European RA- all SAR data. The pre-processing of the multispectral dar-Optical Research Assemblage library - ERA-ORA) optical images includes atmospheric correction and were considered. By the combination of this expert orthorectification using the free accessible C-band knowledge about typical target characteristics (e.g. low SRTM DEM. In the context of multisource image data reflectance of water bodies in the near infrared) and fusion a critical pre-processing step is an accurate co- histogram analyses, it is possible to assess scene-spe- registration of all EO-scenes used. cific threshold values. In the third stage of the proposed classification scheme the identified trainings sites will After pre-processing different image fusion approaches be used as input for a supervised classification. In the were applied. Generally, in literature following methods framework of this study three classification techniques – for the integration of optical and SAR data are com- nearest neighbour, fuzzy logic and a combined pixel- monly applied: combination of both data sets without an /object based approach – were compared. In the latter alteration of the original input channels and image fu- case a pixel-based maximum likelihood classification sion on pixel- and decision level. In the framework of was performed. The final land cover category assigned this study the potential of the first and the second data to each image object corresponds to the most frequent integration approach for land cover and cop type map- class per image segment. Postclassification procedures ping will be assessed. Performed pixel-based fusion involve simple GIS-analysis such as the recoding of techniques include IHS-transformations, principle com- island segments within residential areas. ponent analyses (PCA), a multiplicative approach and wavelet-transformations. The benefit of these proce- Thematic map accuracy of the final land cover products dures was estimated by separability analyses and a com- was assessed by calculating the confusion matrix and parison of the classification accuracies achieved by a the kappa coefficient for fifty randomly distributed simple pixel-based maximum likelihood classification reference points per land cover category. The class (MLK). The Jeffries-Matusita distance (JM), which is membership of each reference target was specified on widely used in the field of remote sensing to determine base of official land information GIS layers and field the statistical distance between two multivariate, Gaus- data. sian distributed signatures, was calculated. The JM

Figure 3. Proposed processing chain

4. RESULTS suitable tool to combine medium resolution optical and SAR data. Perhaps, it will be appropriate to fuse images 4.1 Pixel-based image fusion versus combination of with different spatial resolution such as Envisat ASAR optical and SAR data – a comparison WSM and MERIS data.

In literature two methods for the integration of optical 4.2 Potential of multitemporal SAR data for land and SAR data are commonly applied. Image fusion cover and crop type mapping products were generated on base of Landsat-5 TM (channel 3, 4 and 5) and HV-polarized ASAR APP data A further objective of this paper was to demonstrate the acquired nearly simultaneously on July 10, 2005. The power of multitemporal SAR data for crop type map- results of the multiplicative approach and the wavelet ping issues, as in Central Europe the number of optical transformations (tested filter functions: Haar, Daube- data available is often limited due to frequent cloud chies, Coiflet, symmetric) are visually very similar to cover. For example, over the Nordhausen test site only the original optical data. For other approaches such as two cloudless Landsat-5 TM scenes were acquired be- the PCA fusion product and the HIS transformation, the tween April and mid of August, i.e. during the main SAR information is more pronounced. To assess the growing season. The first image was recorded on April benefit of these approaches for land cover and crop type 22 and the second on July 10. Both acquisition dates are mapping separability analyses were performed. By the not well suited for crop type mapping. In consequence, combination of the optical and SAR data without an the achieved classification accuracies for the monotem- alteration of the pixel values the separability rises sig- poral optical data are not sufficient (Tab. 1). By using nificantly. Contrary, for the images fused on pixel level both Lansat-5 TM scenes available, the accuracy of the the separability remains unchanged or even declines for final land cover map could be improved significantly. all class pairs. These findings were confirmed by the However, the results of the investigations showed that achieved classification accuracies. Using the optical and the classification accuracies obtained on base of the SAR data as independent input layer for a supervised SAR data only exceeds those of the optical data indi- MLK-classification, the overall accuracy increases by cating a high potential of multitemporal radar data for 7.2 % (74.4 % → 81.6 %). Especially for the classes land cover and crop type mapping. To improve the urban areas, grassland, winter wheat, winter barley, peas mapping of urban areas the usage of textural features and corn an improved accuracy could be achieved. The extracted on base of HH-polarized SAR data is recom- land use / crop type maps derived on base of the images mended. In order to reduce misclassifications between fused on pixel level shows no improvement for all forests and agriculture/grassland as well as urban areas classes. For some land use categories the user and pro- and agriculture/grassland the multitemporal minimum in ducer accuracies even declines. HV-polarisation could be used. Regarding the optimal polarization for the derivation of crop type information For the mapping of urban areas several studies demon- the analysis indicated that the cross-polarized data are strated the power of textural features in conjunction most suitable. The final product achieved on base of the with spectral or/and backscatter information using me- ERS-2 data was less accurate, though it has to be con- dium resolution EO-data [7, 8]. In the framework of this sidered that the number of SAR scenes used for classifi- study it was investigated whether it is possible to im- cation was not equal (HV 9 vs. VV 6). This will be prove the extraction of textural features using the pixel- analyzed in more detail in future investigations. Finally, based fusion products as input layer. Previous analyses the results listed in Table 1 showed that by the combi- indicated the potential of the textural measures second nation of optical and SAR data the classification accu- angular moment (SAM) computed on base of the grey racy could be improved. level co-occurrence difference vector (GLDV) for the Landsat-5 TM data as well as the standard derivation The retrieved land cover maps are illustrated in Figure and the neighbourhood grey level dependency matrix 4. Though a simple pixel-based MLK-classification (NGLD) for SAR data [9]. A visual interpretation of the approach was followed, the derived land cover products textural features extracted on base of the pixel-based look very ‘smooth’, especially the results obtained on fusion products indicates a higher separability using the base of the SAR data as well as the optical and SAR fused image. However, this hypothesis was not data. The differentiation of urban and unvegetated areas supported by the achieved classification accuracies. was problematic in all cases. Indeed, by the integration Best overall performance was found using the SAR data of the SAR information in the classification process for texture extraction only. The urban area maps derived misclassifications could be reduced significantly. Not from the fused products as well as from the optical data surprisingly, the results showed that it is not possible to were less accurate and stable in time. distinguish coniferous and deciduous / mixed forests on In conclusion, the investigation indicated that the appli- base of SAR data only. In the east of the test site near cation of pixel-based image fusion procedures is not a the Berga- reservoir meadow and pasture are Excl. HV, HV, HV, HV VV, VV 4.3 Classification scheme for the automated genera- SAR VV, VV tex., tex. tion of basic land cover maps tex., HV-m HV-m Main objective of this study was to set up a classifica- SAR 80.2 77.9 73.3 71.9 64.3 65.2 tion scheme for the derivation of basic land cover cate- TM 2104 52.8 82.8 81.8 80.4 80.4 75.8 76.8 gories with a high potential for automation. As outlined in chapter 3 the second processing step compromises the TM 1007 68.3 82.4 82.7 83.6 82.2 80.1 80.6 selection of potential training samples on base of a deci- TM 2104 77.9 83.7 83.7 84.0 83.8 82.9 82.5 sion tree. The absolute threshold values at each node of & 1007 the decision tree are estimated stepwise making use of Tab. 1. Classification accuracies – 20 land cover expert knowledge in combination with histogram analy- classes, 50 reference points per class – optical (orange), ses. The analyses outlined above demonstrated the util- SAR (green), combination (blue) ity of textural features derived on base of HH-polarised C-band data and the multitemporal minimum in HV- predominant. These grassland areas were detected by polarisation for land cover mapping. Furthermore it was both the optical and SAR data. Regarding the crop types shown, that image fusion at pixel level is not a suitable cultivated in the test site the classification result indi- tool to improve the accuracy of the final land cover cated a low separability of cereals. All classification product. In Table 2 the characteristic image parameters results show a mix-up between winter wheat and sum- used to compute the thresholds for each land cover class mer wheat, winter rye, triticale and oat. However, the are listed. number of fields or/and the mean field acreage of these crops (except winter wheat) is low. Thus, on base of the Exemplary the threshold estimation process is described crop type distribution in the test site in 2005 it is not in detail for the land cover category coniferous forest. In possible to draw a conclusion on the potential of agreement with the general knowledge of reflectance multitemporal C-band SAR data (and Landsat-5 TM properties, the analyses of the time series available as data) for the differentiation of cereals.

Water Triticale

Coniferous forest Summer barley

Deciduous/mixed forest Summer wheat

Urban areas Oat

Unvegetated areas Rape

Grassland Corn

Clover Peas

Winter wheat Hop

Winter barley Potatoes

Winter rye Sugar beets

Fig. 4. Land use map – combination optical & SAR (upper left), optical (upper right) and SAR (bottom)

Class Image Parameter agreement with the reference data, usually exceeds Water bodies NIR 90%. However, especially for grassland problems in Coniferous forest NIR, MIR, NDVI finding correct training samples arise when monotempo- Dec. / mixed f. - leafoff Green, NDVI, tex. – HH-pol. ral optical data are available only. Dec. / mixed f. - leafon Green, NIR - MIR Unvegetated areas Multitemp. min. red, HH, HV For classification three different approaches were com- Urban areas NIR, MIR, tex. – HH-pol., min. HV-pol. pared considering the training sites and threshold values Grassland NDVI, HV-pol. specified by the methodology described above. First, a Tab. 2. Characteristic image parameter used for the simple nearest neighbour classification was performed automatic selection of training samples using all potential training samples as input data. The second classification procedure implies the usage of the well as of the EO-libraries showed that coniferous forest class thresholds estimated to set up an object-based areas are characterized by a low near and middle infra- fuzzy classification rule. Finally, a combined pixel- red reflectance. First of all, segments most probably based/ object-based approach was applied. The Landsat- belonging to the water class were excluded by un- TM 5 scenes acquired on April 22 and July 10, 2005 as selecting all image objects with a reflectance in NIR well as multitemporal ASAR APP data from 2005 were below the threshold for water. In the next step an initial used as input for classification. The obtained map accu- threshold for coniferous forest in MIR is defined as the racies differ significantly (Tab. 3), whereby the best per- minimum histogram value with a segment frequency formance was found by far for the combined pixel- greater than five plus twenty. By this process mainly /object-based classification. coniferous forest and - depending on growth stage – agricultural crops are selected. Both classes could be Class Nearest Fuzzy Combined neighbour classification pixel-/object separated analyzing the corresponding histogram in NIR (Fig. 5). Thereby the lower peak represents coniferous PA UA PA UA PA UA forest segments and the higher one agricultural fields. Water 98.0 100.0 95.8 100.0 96.0 100.0 Applying the threshold estimated for the NIR channel, Coniferous f. 67.4 100.0 91.3 100.0 100.0 92.5 the final threshold in MIR could be estimated by histo- Mixed/dec. f. 90.0 73.8 100.0 89.1 92.0 95.8 gram analyses. In the end, as sometimes a very small Grassland 60.0 93.8 69.4 85.0 96.0 88.9 number of urban objects are selected, the mean NDVI ± Unveg. areas 36.0 100.0 48.9 92.0 74.0 97.4 0.05 is computed and used for the selection of the final Urban areas 81.3 78.0 75.0 83.7 84.0 72.4 training samples of coniferous forest. Agriculture 97.9 46.1 97.7 53.8 89.6 89.6 Overall acc. 75.6 82.5 90.2 Kappa 0.715 0.796 0.886 Tab. 3. Classification accuracies - TM acquired in April and July 2005 and multitemporal SAR data from 2005 (PA – producer accuracy; UA – user accuracy)

For validation issues the proposed methodology was tested using different sets of input data including images acquired in 2003 and 2005 (Tab. 4). The corresponding classification accuracies are listed in Table 5. Not sur- prisingly, the obtained land cover maps are less accurate in comparison to those generated on base of multitem- poral optical data. Stable classification accuracies were achieved for the land cover classes water, coniferous Figure 5. Threshold estimation by histogram analysis – forest, deciduous/mixed forest and urban areas. As mo- example coniferous forest notemporal optical data were used for classification only, increased misclassifications occur between un- By the proposed methodology a large number of poten- tial trainings sites will be selected. For validation issues Landsat- ASAR APP IS2 – HH/HV the algorithm was tested for all EO-data available. The TM/ETM Date 1 Date2 Date3 quality of the training samples selected was assessed by Set 1 21.04.05 22.04.05 10.07.05 24.08.05 a visual inspection and the calculation of the confusion Set 2 10.07.05 22.04.05 14.08.05 18.09.05 matrix on base of the reference data used for the accu- Set 3 06.08.03 11.06.03 20.08.03 14.09.03 racy assessment of the final land cover map. The user Set 4 22.04.03 16.07.03 20.08.03 14.09.03 accuracy, i.e. the probability that a training sample is in Tab.4 Input data sets used for validation Set1 Set2 Set3 Set4 measures extracted on base of TerraSAR-X and PAL- PA UA PA UA PA UA PA UA SAR ALOS data will most probably improve the gen- WA 96.0 96.0 94.0 100. 98.0 100. 96.0 98.0 eration of residential area maps. CF 100 94.2 100. 94.2 95.9 92.2 95.4 95.4 D/MF 94.0 97.9 92.0 79.3 92.0 93.9 96.0 92.3 6. ACKNOWLEDGEMENT GL 56.0 93.3 54.0 87.1 92.0 76.7 84.0 63.6 UA 68.0 94.4 62.0 81.6 72.0 83.7 64.0 94.1 UR 78.0 86.7 82.0 74.6 80.0 81.6 83.0 78.0 The ENVISAT ASAR and ERS-2 data were provided AG 97.9 54.7 91.7 66.7 81.3 84.8 58.3 63.6 courtesy of the European Space Agency (Category-1 OA 84.2 82.1 87.3 82.3 Project C1P 3115). The Enviland project – subproject KA 0.81 0.79 0.85 0.80 scale integration - is funded by the German Ministry Tab. 5. Classification results - validation Economy and Technology (MW) and the German Aero- space Centre (DLR) (FKZ 50EE0405). vegetated areas, grassland and agricultural fields. In consequence, the producer and user accuracies of the 7. REFERENCES corresponding land use categories decline. 1. Alparone, L., Baronti, S., Garzelli, A. & Nencini, F. 5. CONCLUSIONS AND OUTLOOK (2004). Landsat ETM+ and SAR image fusion based on generalized intensity modulation. IEEE In the first section of this paper two common methods Trans. Geosc. RS, 42(12): 2832 – 2839. for the integration of optical and SAR data were com- 2. Amarsaikhan, D. & Douglas, T. (2004). Data fusion pared. The investigation showed that a simple combina- and multisource image classification. Int. J. RS tion of the information provided by both data types is 25/17: 3529 – 3539. more suitable to improve land cover map accuracy than 3. Hegarat-Mascle S L, Quesney A, Vidal-Madjar D, image fusion at pixel level. By the combination of opti- Taconet O, Normand M & Loumagne (2000). cal and radar images a significant rise in product accu- Land cover discrimination from multitemporal racy could be achieved. Promising classification results ERS images and multispectral Landsat images: a were obtained using multitemporal SAR data only. The study case in an agricultural area in France. Int. J. classification accuracy exceeds those obtained on base RS, 21: 435 – 456. of two Landsat-5 TM scenes acquired in April and July. 4. Pohl, C. & Van Genderen, J. L. (1998). Multisen- sor image fusion in remote sensing: concepts, me- Finally, a combined object-/pixel-based classification thods and applications. Int. J. RS 19/5: 823 – 854. scheme for the generation of basic land cover maps 5. Stussi N., A. Beaudoin, T. Castel & P. Gigord providing a high potential for automation has been pre- (1995). Radiometric correction of multiconfigura- sented. The training samples used as input for a super- tion spaceborne SAR data over hilly terrain. In: vised classification were selected on base of an object- Proc. Int. Symp. on Retr. of Bio- and Geophy. based decision tree with scene-specific, flexible thresh- Para. from SAR Data for Land Appl., Toulouse, olds. The absolute threshold values were calculated by France, 10-13 October 1995, pp. 469-478. the combination of expert knowledge and histogram 6. Baatz, M. & Schäpe, A. (2000). Multiresolution analyses. In order to improve the mapping accuracy for Segmentation– an optimization approach for high urban areas textural features extracted from HH-polar- quality multi-scale image segmentation. In: Strobl ized SAR data were incorporated in the classification J et al. (Eds.): Angew. Geogr. Informationsverar- procedure. For validation issues the methodology was beitung XII. Beiträge zum AGIT-Symposium applied to different sets of input data acquired over the Salzburg, Herbert Wichmann Verlag, pp. 12–23. Nordhausen test site in 2003 and 2005. 7. Dekker, R. J. (2003). Texture analysis and classifi- cation of ERS SAR images for map updating of ur- With the availability of polarimetric and high-resolution ban areas in The Netherlands. IEEE Trans. Geosc. spaceborne X-and L-band SAR data as provided by the RS, 41(9):1950 – 1958. TerraSAR-X and PALSAR ALOS mission a further 8. Dell’ Acqua, F., Gamba, P. (2003). Texture-based improvement is expected for both the automatic selec- characterization of urban environments on satellite tion of potential training samples as well as the final SAR images. IEEE Trans. on Geosc. RS, 41(1): map accuracy. For example, L-band SAR data are 153 – 159. known to provide an excellent database for forest cover 9. Riedel, T., Thiel, C. & Schmullius, C. (2006): An mapping, i.e. the misclassification between forested object-based classification procedure for the areas, agricultural crops and grassland are expected to derivation of broad land cover classes using optical decrease. Regarding urban area mapping several studies and SAR data. In: Proc. Object-Based Image emphasized the potential of X- und L-band data. Fur- Processing (OBIA), Salzburg, Austria, July 4-5, thermore, due to the very high spatial resolution, texture 2006, on CD.