computers Article A Comparison of Compression Codecs for Maritime and Sonar Images in Bandwidth Constrained Applications Chiman Kwan 1,* , Jude Larkin 1 , Bence Budavari 1, Bryan Chou 1, Eric Shang 2 and Trac D. Tran 3 1 Applied Research, LLC, Rockville, MD 20850, USA; [email protected] (J.L.); [email protected] (B.B.); [email protected] (B.C.) 2 Leidos, Columbia, MD 21046, USA; [email protected] 3 Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA; [email protected] * Correspondence: [email protected]; Tel.: +1-240-207-2311 Received: 14 February 2019; Accepted: 24 April 2019; Published: 28 April 2019 Abstract: Since lossless compression can only achieve two to four times data compression, it may not be efficient to deploy lossless compression in bandwidth constrained applications. Instead, it would be more economical to adopt perceptually lossless compression, which can attain ten times or more compression without loss of important information. Consequently, one can transmit more images over bandwidth limited channels. In this research, we first aimed to compare and select the best compression algorithm in the literature to achieve a compression ratio of 0.1 and 40 dBs or more in terms of a performance metric known as human visual system model (HVSm) for maritime and sonar images. Our second objective was to demonstrate error concealment algorithms that can handle corrupted pixels due to transmission errors in interference-prone communication channels. Using four state-of-the-art codecs, we demonstrated that perceptually lossless compression can be achieved for realistic maritime and sonar images. At the same time, we also selected the best codec for this purpose using four performance metrics. Finally, error concealment was demonstrated to be useful in recovering lost pixels due to transmission errors. Keywords: perceptually lossless compression; error recovery; maritime and sonar images; JPEG2000; X264; X265; Daala 1. Introduction Since the appearance of the Joint Photographic Experts Group (JPEG) standard in the 1990s, image compression has been well-developed [1–6]. Lossless image compression algorithms include JPEG [7] and JPEG2000 [8]. Some recent algorithms such as X264 (software implementation of H.264/AVC standard) [9] and X265 (software implementation of H.265/HEVC standard) [10] also provide lossless compression options. JPEG, X264, and X265 are discrete cosine transform (DCT) based algorithms and JPEG2000 is wavelet based. About 15 years ago, there were some developments in DCT based algorithms, where overlapped blocks known as lapped transforms (LT) were used to further improve the compression [11]. In the past few years, a group of researchers at Xiph have incorporated LT [11] into an open source codec known as Daala [12]. Through several years of rigorous development, Daala has reached a stage where it outperforms X264 and has comparable performance to X265 in terms of a performance metric that mimics human visual system (HVS) [12,13]. In this research, our sponsor had specified three requirements on the image codec for maritime and sonar images: (1) 10 to 1 compression ratio; (2) the decompressed image should have 40 dBs or Computers 2019, 8, 32; doi:10.3390/computers8020032 www.mdpi.com/journal/computers Computers 2019, 8, 32 2 of 21 more in HVSm, which may be considered as “near perceptually lossless” performance; and (3) the lost pixels need to be effectively concealed without incurring additional bandwidth. It should be noted that “perceptually lossless” has been defined in digital picture coding since mid-1990s [14]. One conventional way to handle transmission errors is to adopt error correction coding, which will add redundant bits and reduce the data transmission efficiency. We explored error concealment techniques, which do not incur additional bandwidth usage. It is important to emphasize that perceptual performance requires a suitable metric. Some conventional metrics such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) may not match well with human’s subjective evaluations. Two metrics in the literature that have better correlation with human perception in the literature were also compared in our studies. In the literature, there are excellent papers discussing perceptually lossless coding [15–18]. The authors of [15] provided a survey of various ideas of embedding human perceptual model into the encoding process of various codecs such as JPEG2000, H264, and H265. The concepts of just noticeable difference (JND) and just not noticeable difference (JNND) [15–18] were introduced and embedded into some of the codecs. The output bit stream is compliant with the standards. Daala also incorporated perceptual vector quantization (PVQ) into its codec [12,13], which resulted in enhanced performance. We would like to emphasize that our paper is not about new developments in the perceptually lossless coding. In our studies, we used the codecs using the default settings and did not change any of the parameters. The only parameters we varied is the quality (or quantization) parameter (qp). The decompressed images were then evaluated using the four metrics, two of which are related to HVS. Similar to some existing papers (e.g., [15]), our sponsor specified 40 dBs or more in terms of HVSm, which may be considered as “near perceptually lossless”. It should be noted that this paper is an extension of our conference paper [19]. There are several key differences. First, we corrected an issue related to the compression ratio computation in our earlier study [19]. Since the input image formats for different codecs are different, we converted all the raw images to different image formats in our earlier study. For instance, X264 and X265 require the format to be Y4M and JPEG2000 and Daala use other formats. However, one issue is that the converted file sizes are slightly different for different formats. In our earlier study [19], each codec was using its own set for computing the compression ratios, causing some slight differences in terms of compression ratios. In this study, we consistently used the original image sizes as references instead of the converted ones. Second, when we generated the results presented in [19], X264 and X265 could only handle YUV420 format. In this study, we used the latest versions of X264 and X265, which can handle YUV444 format. Moreover, Daala has a newer version, which is improved over its previous version. Now, we re-generated all the performance metrics for all the datasets in this study. Third, we added two new sections to this paper containing new maritime and sonar images, and one new section containing images from the Xiph website. This is to demonstrate that the compression algorithms need to be robust to images with different resolutions and modalities. Fourth, we also present new error concealment results for the maritime and sonar images in this paper. Again, our goal was to demonstrate that the error concealment algorithms can handle different image modalities. Although the compression algorithms are well-known, there was still some customization work in this project. For example, the use of YUV444 format instead of the default YUV420 ensured high quality compression for X264 and X265 for still images. Moreover, the error concealment needed to be tailored to the data that we used. One key contribution of our project was to integrate three components (compression, error concealment, and near perceptually lossless evaluation after decompression) in the compression process into a single system to achieve 10 to 1 near perceptually lossless compression for maritime and sonar images. That is, the novelty of our paper is not in new perceptually lossless compression theory, but rather in the integration of existing compression and error concealment technologies to achieve near or even perceptually lossless image compression at 10 to 1 compression for bandwidth constrained and interference-prone applications. Computers 2019, 8, 32 3 of 21 Our paper is organized as follows. Section2 summarizes the technical approach, the various codecs, and the performance metrics. Section3 summarizes all the experiments using actual maritime and sonar images that are of interest to our customer. Visual comparisons of different codecs at 10 to 1 compression are also included. Finally, concluding remarks are given in Section4. Computers 2019, 2, x FOR PEER REVIEW 3 of 20 2. Technical Approach 2. Technical Approach 2.1. Proposed Image Compression and Error Recovery Approach 2.1. Proposed Image Compression and Error Recovery Approach In this study, we focused on objective evaluations using four well-known compression algorithms in theIn literature. this study, This we was focused to ensure on that objective we delivered evaluations the best using algorithm four to well-known our customer. compression Our overall technicalalgorithms approach in the literature. is summarized This was in Figure to ensure1. First, that we we briefly delivered reviewed the best the state-of-the-art algorithm to our compression customer. algorithmsOur overall available technical on approach the market. is summarized At the same in time, Figure we 1. described First, we dibrieflyfferent reviewed performance the state-of-the- metrics for algorithmart compression evaluation. algorithms The focus available was on on metrics the mark thatet. can At better the same model time, human we perception.described different We also reviewedperformance the metrics error