1 Contrast Enhancement Estimation for Digital Image 2 Forensics 3 4 GE Global Research Center, USA 5 LONGYIN WEN, 6 HONGGANG QI, University of Chinese Academy of Sciences, China 7 SIWEI LYU, University at Albany, State University of New York, USA and Tianjin Normal University, China 8 Inconsistency in contrast enhancement can be used to expose image forgeries. In this work, we describe a 9 new method to estimate contrast enhancement operations from a single image. Our method takes advantage 10 of the nature of contrast enhancement as a mapping between pixel values, and the distinct characteristics it 11 introduces to the image pixel histogram. Our method recovers the original pixel histogram and the contrast 12 enhancement simultaneously from a single image with an iterative algorithm. Unlike previous works, our 13 method is robust in the presence of additive noise perturbations that are used to hide the traces of contrast 14 enhancement. Furthermore, we also develop an effective method to detect image regions undergone contrast 15 enhancement transformations that are different from the rest of the image, and use this method todetect composite images. We perform extensive experimental evaluations to demonstrate the efficacy and efficiency 16 of our method. 17 18 CCS Concepts: • Mathematics of computing → Convex optimization; • Computing methodologies → 19 Scene anomaly detection; • Applied computing → Evidence collection, storage and analysis; 20 Additional Key Words and Phrases: Media Forensics, Contrast Enhancement, Pixel Histogram 21 22 ACM Reference Format: 23 Longyin Wen, Honggang Qi, and Siwei Lyu. 2017. Contrast Enhancement Estimation for Digital Image 24 Forensics. ACM Trans. Multimedia Comput. Commun. Appl. 0, 0, Article 00 ( 2017), 18 pages. https://doi.org/ 25 0000001.0000001 26 27 1 INTRODUCTION 28 The integrity of digital images has been challenged by the development of sophisticated image 29 editing tools (e.g., Adobe Photoshop), which can modify contents of digital images with minimal 30 visible traces. Accordingly, the research field of digital image forensics [11] has experienced rapid 31 developments in the past decade. Important cues to authenticate digital images and detect tamper- 32 ing can be found from various steps in the image capture and processing pipeline, and one of such 33 operations is contrast enhancement. Contrast enhancement is a nonlinear monotonic function of 34 pixel intensity, and it is frequently exploited to enhance image details of over-or under-exposed 35 regions. Commonly used contrast enhancement transforms include gamma correction, sigmoid 36 37 Authors’ addresses: Longyin Wen, GE Global Research Center, 1 Research Circle, Niskayuna, NY, 12309, USA; Honggang Qi, 38 University of Chinese Academy of Sciences, School of Computer and Control Engineering, No.3, ZhongGuanCun NanYiTiao, Beijing, 100049, China; Siwei Lyu, University at Albany, State University of New York, Computer Science Department, 1400 39 Washington Avenue, Albany, NY, 12222, USA, Tianjin Normal University, School of Computer and Information Engineering, 40 No.393, Binshui XiDao, Tianjin, China. 41 42 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee 43 provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice andthe 44 full citation on the first page. Copyrights for components of this work owned by others than the author(s) must behonored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires 45 prior specific permission and/or a fee. Request permissions from [email protected]. 46 © 2017 Copyright held by the owner/author(s). Publication rights licensed to the Association for Computing Machinery. 47 1551-6857/2017/0-ART00 $15.00 48 https://doi.org/0000001.0000001 49 ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 0, No. 0, Article 00. Publication date: 2017. 00:2 Longyin Wen, Honggang Qi, and Siwei Lyu 50 stretching and histogram equalization [13]. In digital image forensics, recovering contrast enhance- 51 ment is useful to reconstruct the processing history of an image. Also, detecting regions undergone 52 different contrast enhancement can be used to expose a composite image. 53 There have been several methods to estimate contrast enhancement from an image [5, 6, 10, 16, 54 19, 20, 23]. However, these methods have two main limitations. First, most of these algorithms are 55 designed for a specific type of contrast enhancement transform (e.g., gamma correction). Second, 56 these algorithms in general lack robustness with regards to noise perturbations that are added to 57 hide the traces of contrast enhancement [1,7]. 58 In this work, we describe a general method to recover contrast enhancement from a single image. 59 Our method exploits the observations that: (i) although contrast enhancement is typically a nonlin- 60 ear function of pixel values, it is a linear transformation of pixel histogram; (ii) pixel histogram after 61 contrast enhancement tends to have more empty bins; and (iii) the effect of additive noise corre- 62 sponds to a convolution of pixel histogramwith the noise distribution. Accordingly, we formulate 63 the estimation of contrast enhancement as an optimization problem seeking recovered pixel his- 64 togram to be consistent with the observed pixel histogram after contrast enhancement transform 65 is applied, while with minimum number of empty bins. The original problem is intractable, so we 66 further provide a continuous relaxation and an efficient numerical algorithm to solve the relaxed 67 problem. Our formulation can handle the estimation of parametric and nonparametric contrast en- 68 hancement transforms, and is robust to additive noise perturbations. Furthermore, we also develop 69 an effective method to detect regions undergone contrast enhancement operation different from 70 the remaining of the image, and use this method to detect composite images generated by splicing. 71 A preliminary version of this work was published in [22]. The current work extends our pre- 72 vious method in several key aspects. First, this work uses a more stable property of contrast 73 enhancement on pixel histogram based on the number of empty bins, which leads to a new type 74 of regularizer in the optimization objective. Furthermore, we include a new optimization method 75 based on the Wasserstein distance and augmented the original algorithm to handle additive noises. 76 The optimization of the overall problem is solved with a more efficient projected gradient descent 77 method. 78 The rest of this paper is organized as follows. In Section2, we review relevant previous works. In 79 Section3, we elaborate on the relation of contrast enhancement and pixel histogram, and describe 80 our algorithm estimating parametric and nonparametric contrast enhancement transforms. In 81 section4, we present the experimental evaluations of the global contrast enhancement estimation 82 algorithm. Section5 focuses on a local contrast enhancement estimation algorithm based on the 83 global contrast enhancement estimation algorithm and graph cut minimization. Section6 concludes 84 the article with discussion and future works. 85 86 2 BACKGROUND AND RELATED WORKS 87 2.1 Contrast Enhancementas Linear Operator on Pixel Histogram 88 89 Our discussion is for gray-scale images of b bit-pixels. A (normalized) pixel histogram represents b 90 the fractions of pixels taking an individual value out of all 2 different grayscale values, and is b 91 usually interpreted as the probability distribution of a random variable X over f0; ··· ;n = 2 − 1g. 92 A contrast enhancement is a point-wise monotonic transform between pixel values i; j 2 93 f0; ··· ;ng, defined as i = ϕ¹jº := »m¹iº¼, where m¹·º : »0;n¼ 7! »0;n¼ is a continuous non-decreasing 94 function, and »·¼ is the rounding operation that maps a real number to its nearest integer. 95 There are two categories of contrast enhancement transforms. A parametric contrast enhance- 96 ment transform can be determined with a set of parameters. An example of parametric contrast 97 enhancement transform is gamma correction, 98 ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 0, No. 0, Article 00. Publication date: 2017. Contrast Enhancement Estimation for Digital Image Forensics 00:3 γ 99 i j = ϕγ ¹iº := n ; (1) 100 n 101 where γ ≥ 0 is the parameter controlling the shape of the transform. Another often-used parametric 102 contrast enhancement transform is sigmoid stretching, − − 103 2 S i nµ − S nµ 3 104 6 © nα nα ª7 j = ϕα; µ ¹iº := 6n 7 ; (2) 6 ­ ¹ − º − ®7 105 6 ­S n 1 µ − S nµ ®7 6 nα nα 7 106 4 « ¬5 2 » ¼ ¹ º 1 107 where α > 0 and µ 0; 1 are two parameters, and S x = 1+exp(−xº is the sigmoid function. On 108 the other hand, a nonparametric contrast enhancement transform affords no simple parametric 109 form and has to be specified for all i 2 f0; ··· ;ng. An example of the nonparametric contrast 110 enhancement transform is histogram equalization, which maps the pixel histogram of an image to 111 match a uniform distribution over f0; ··· ;ng. ˜ ˜ 112 Consider two images I and I, with the pixels of I obtained from those of I using a contrast 113 enhancement transform ϕ. We introduce two random variables X;Y 2 f0; ··· ;ng as the pixels 114 of I and I˜, hence Y = ϕ¹Xº. Using the probability interpretation of pixel histogram, the contrast 115 enhancement transform between X and Y induces the conditional probability distribution Pr¹Y = 116 jjX = iº = 1j=ϕ¹iº, where 1c is the indicator function whose output is 1 if c is true and zero otherwise. As such, we have 117 Í Pr¹Y = jº = i 1j=ϕ¹iº Pr¹X = iº: (3) 118 We can model a pixel histogram as a vector on the n-dimensional probability simplex, i.e., h 2 119 n+1 T ∆ := fhjh ⪰ 0; 1 h = 1g with hi+1 = Pr¹X = iº for i = 0; ··· ;n, and a contrast enhancement ϕ 120 as an ¹n + 1º × ¹n + 1º matrix Tϕ : Tϕ = 1j=ϕ¹iº.