applied sciences Article Automatic Shadow Detection for Multispectral Satellite Remote Sensing Images in Invariant Color Spaces Hongyin Han 1,2 , Chengshan Han 1, Taiji Lan 1, Liang Huang 1,2, Changhong Hu 1 and Xucheng Xue 1,* 1 Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; [email protected] (H.H.); [email protected] (C.H.); [email protected] (T.L.); [email protected] (L.H.); [email protected] (C.H.) 2 University of Chinese Academy of Sciences, Beijing 100049, China * Correspondence: [email protected]; Tel.: +86-0431-8617-8588 Received: 20 August 2020; Accepted: 13 September 2020; Published: 17 September 2020 Abstract: Shadow often results in difficulties for subsequent image applications of multispectral satellite remote sensing images, like object recognition and change detection. With continuous improvement in both spatial and spectral resolutions of satellite remote sensing images, a more serious impact occurs on satellite remote sensing image interpretation due to the existence of shadow. Though various shadow detection methods have been developed, problems of both shadow omission and nonshadow misclassification still exist for detecting shadow well in high-resolution multispectral satellite remote sensing images. These shadow detection problems mainly include high small shadow omission and typical nonshadow misclassification (like bluish and greenish nonshadow misclassification, and large dark nonshadow misclassification). For further resolving these problems, a new shadow index is developed based on the analysis of the property difference between shadow and the corresponding nonshadow with several multispectral band components (i.e., near-infrared, red, green and blue components) and hue and intensity components in various invariant color spaces (i.e., HIS, HSV, CIELCh, YCbCr and YIQ), respectively. The shadow mask is further acquired by applying an optimal threshold determined automatically on the shadow index image. The final shadow image is further optimized with a definite morphological operation of opening and closing. The proposed algorithm is verified with many images from WorldView-3 and WorldView-2 acquired at different times and sites. The proposed algorithm performance is particularly evaluated by qualitative visual sense comparison and quantitative assessment of shadow detection results in comparative experiments with two WorldView-3 test images of Tripoli, Libya. Both the better visual sense and the higher overall accuracy (over 92% for the test image Tripoli-1 and approximately 91% for the test image Tripoli-2) of the experimental results together deliver the excellent performance and robustness of the proposed shadow detection approach for shadow detection of high-resolution multispectral satellite remote sensing images. The proposed shadow detection approach is promised to further alleviate typical shadow detection problems of high small shadow omission and typical nonshadow misclassification for high-resolution multispectral satellite remote sensing images. Keywords: shadow detection; invariant color space; high-resolution multispectral satellite remote sensing image; threshold; WorldView-3; WorldView-2 1. Introduction More complex details of land covers (e.g., buildings, towers, vegetation, farms and roads) are obtained easily from high spatial resolution (HSR) multispectral satellite remote sensing images which Appl. Sci. 2020, 10, 6467; doi:10.3390/app10186467 www.mdpi.com/journal/applsci Appl. Sci. 2020, 10, 6467 2 of 25 are captured by the recently launched HSR satellites (like IKONOS, GeoEye-1, QuickBird, WorldView-2, WorldView-3 and Jilin-1) [1–7]. However, shadow inevitably formed by land objects and clouds affects more seriously for these HSR image applications, such as change detection, object recognition and image classification. Additional cues are obtained from the HSR images with the palpable shadow, such as the general shape and structure of cast objects, the illumination direction and the position of the sun, as well as parameters of the satellite sensor. These cues are also helpful in numerous applications, like building detection, height estimation, 3D reconstruction, change surveillance, scene interpretation and position estimation of the sun and satellites [4,5,8–12]. On the other hand, shadow in HSR images may cause serious shape distortion of cast objects, false color tone and loss of feature information, which may result in negative effects in subsequent image applications [13]. Given either the useful or troublesome influence of shadow in HSR multispectral satellite remote sensing images, in order to improve the utilization of HSR multispectral satellite remote sensing images, shadow detection is an important scientific issue for HSR multispectral remote sensing images, which is usually the first step followed by shadow compensation and image utilization [9,12,14]. Much research has been developed on shadow detection for both color aerial images and multispectral satellite remote sensing images in recent decades. Huang et al. [15] proposed a shadow detection method through developing an imaging model indicating the increased amount of hue values in shadow regions compared with the corresponding nonshadow ones. A certain threshold was employed to obtain shadow candidate in accordance with the increased hue values in shadow regions, and two other thresholds were subsequently used with respect to blue (B) and green (G) components to refine the shadow candidate by eliminating greenish and bluish nonshadow objects. Huang et al. developed a useful imaging model and the deduced shadow detection algorithm was firstly dedicated to resolving the bluish and greenish nonshadow object misclassification problem in color aerial images, even though thresholds were selected manually. Moreover, Sarabandi et al. [16] proposed a C3 shadow detection method by studying the shadow identification results of both IKONOS and QuickBird multispectral images through C1, C2 and C3 components in the color space C1C2C3, respectively. The C3-based algorithm could identify the broad outline of large shadow regions. However, most greenish nonshadow objects were misclassified. Similarly, Arevalo et al. [17] presented a semi-automatic shadow detection algorithm built on the C3 component of C1C2C3 color space and a region-growing procedure for HSR pan-sharpening satellite remote sensing images. Comparative experiments revealed that the presented shadow detection approach achieved higher accuracies and better robustness against the RGB-based algorithm by Huang et al. [15] and the C3-based algorithm by Sarabandi et al. [16] Considering all available bands of the multispectral image, Besheer et al. [18] proposed a modified C3 (MC3) index through developing an improved C1C2C3 invariant color space by employing the near-infrared (NIR) band information in addition to visible bands (i.e., red (R), green and blue bands) in the original C1C2C3 invariant color space. Then, the shadow was segmented with a bimodal histogram threshold. The MC3 method delivered an improved performance by picking up the NIR component into consideration in contrast to the C3 method by Sarabandi et al. [16] and Arevalo et al. [17]. Additionally, based on the Huang’s imaging model [15] and the Phong illumination model [19], Tsai [20] presented an automatic property-based shadow detection approach utilizing the ratio of hue value over intensity value, called the spectral ratio index (SRI) shadow detection method. Subsequently, the Otsu thresholding method [21] was used to determine an optimal threshold automatically. The SRI algorithm was tested with comparative studies in various invariant color spaces (HIS, HSV, HCV, YIQ and YCbCr) for color aerial images. The comparative results showed that the SRI shadow detection approach drew higher shadow detection accuracies in HIS, YIQ and YCbCr color spaces, though some greenish grass in nonshadow regions was still misclassified more or less. Subsequently, Khekade et al. [22] further enhanced the shadow detection results of the SRI algorithm by Tsai [20] particularly in the YIQ invariant color space by using a series of post-processing methods (e.g., histogram equalization and box filter). Comparative experiments in color aerial images against Appl. Sci. 2020, 10, 6467 3 of 25 the original SRI images of Tsai showed that the enhanced shadow detection method improved the shadow omission problem in the visual aspect. On the foundation of Tsai’s efficient shadow detection algorithm, Chung et al. [23] proposed a modified ratio map by applying an exponential function to the SRI by Tsai, and presented a successive thresholding scheme (STS) rather than only using a global threshold [20]. Experiments in color aerial images revealed that the proposed algorithm by Chung et al. [23] showed an improved performance in detecting shadow in images containing low brightness objects. Inspired by the STS procedure by Chung et al. [23]. Silva et al. [24] extended the SRI method by Tsai [20] specifically in the CIELCh color space by applying a natural logarithm function to the original ratio map to compress the original values, resulting in the logarithmic spectral ratio index (LSRI) algorithm. Then, the ratio map was segmented by applying multilevel thresholding. This modified ratio method performed better in color aerial images by accurately detecting shadow and avoiding misclassifying dark areas compared with the original ratio method by Tsai [20]