Light Pollution Reduction in Nighttime Photography
Total Page:16
File Type:pdf, Size:1020Kb
Light Pollution Reduction in Nighttime Photography Chang Liu Xiaolin Wu Shanghai Jiao Tong University McMaster University Abstract Nighttime photographers are often troubled by light pol- lution of unwanted artificial lights. Artificial lights, after scattered by aerosols in the atmosphere, can inundate the starlight and degrade the quality of nighttime images, by reducing contrast and dynamic range and causing hazes. In this paper we develop a physically-based light pollution reduction (LPR) algorithm that can substantially alleviate the aforementioned degradations of perceptual quality and restore the pristine state of night sky. The key to the suc- cess of the proposed LPR algorithm is an inverse method to estimate the spatial radiance distribution and spectral sig- nature of ground artificial lights. Extensive experiments are Figure 1: First column: light-polluted images. Second col- carried out to evaluate the efficacy and limitations of the umn: restored images by the proposed LPR algorithm. LPR algorithm. requires to model the image formation process I^ = F (I;J), where I is the ideal image free of interference of artificial ^ 1. Introduction lighting J, and I is the image acquired in presence of J, and solve the inverse problem of recovering I from I^. The above A side effect of urbanization is wide spread of nighttime stated modeling and algorithmic problem of removing light light pollution caused by pervasive artificial lighting and in- pollution in nighttime photography is the main theme and creased density of aerosols in the atmosphere. As light pol- contribution of this paper. We succeed in designing the al- lution distorts the energy level and spectral signature of nat- gorithm and achieving our design goal as can be previewed ural light in the night, it degrades the quality of nighttime in Fig. 1. The ability to image nighttime beauty of pristine images. For example, nowadays it is becoming increasingly nature or sophisticatedly-lit man-made structures is much difficult to capture the Milky Way with a camera; enthusi- desired in many existing and potential applications, such astic night sky photographers are known to go great dis- as visual arts, high dynamic range imaging, environment tances just to escape the city lights. But not everyone has study, and astronomy. To the best of our knowledge, we are arXiv:2106.10046v1 [cs.CV] 18 Jun 2021 the means and time to travel to a location free of artificial the first to attack the problem of light pollution reduction lighting. Even a weak level of light pollution can ruin artis- (LPR) for nighttime photography. tic appeal of night sky photos, because long exposure re- Some previous publications on the subject of light pol- quired to capture distant faint stars will also accumulate the lution are about its adverse effects on the astronomical ob- small amount of artificial lighting to a noticeable level of servations [7, 23]. Other papers discuss about the impact of greyish/brownish background. In addition, light pollution light pollution on human health and environment [3, 4, 8]. may be a hindrance to nighttime photography of city scenes In the field of computer graphics, Jensen et al. studied the as well. For example, a desired image composition requires problem of realistically rendering night sky images [15]. shooting far away illuminated buildings or other structures Their work is based on physically modeling nighttime il- at a spot where nearby street lighting cannot be escaped. lumination effects of astronomical bodies, assuming zero As light pollution problem cannot be physically cor- artificial lighting. rected, the only solution is to algorithmically neutralize un- In the perspective of image restoration, most relevant wanted effects of light pollution on nighttime photos. This to this work is the subject of image dehazing, which has been extensively researched, including traditional image processing algorithms [5, 11], deep learning based algo- rithms [18, 27, 32], and some algorithms especially for nighttime dehazing [16, 31]. The task of light pollution re- duction differs from dehazing in two aspects. Firstly, the degree of light pollution is spatially nonuniform, depending on the geographical distribution and varying strength of arti- ficial lights, and also on how the energy of artificial lighting attenuates in altitude. The mechanism of light scattering in hazy weather is simpler to model as the sun light can Figure 2: The formation process of light-polluted images. be considered of uniform strength in atmosphere and hav- point, λ is the wavelength, and β(λ) is the scattering co- ing a white spectrum. Secondly, the original signal strength efficient, which accounts for the ability of a unit volume in nighttime images is much weaker than in day time im- of atmosphere to scatter light of wavelength λ in all direc- ages. The low signal-to-noise ratio makes the restoration tions [19, 20]. For point light sources that radiate isotrop- task more difficult in the former case than in the latter case. ically like the street lights with respect to atmosphere, the 2. Problem background above attenuation model should be modified to incorporate the inverse-square law, The recovery of light pollution free nighttime images is E (λ)e−βλd an inverse problem stated below: E(d; λ) = 0 ; (3) d2 I^ = I + J (1) 3. Baseline method ^ where I is the light-polluted image captured by camera, I By light pollution of nighttime images we mean the un- is the pristine nighttime image that could only be acquired wanted effects of ground artificial lights being scattered by in total void of artificial lights by a perfectly static camera aerosols in atmosphere. To remove visual effects of light with long exposure, and J is the jamming image formed pollution, we need to model and compute the light pollu- by artificial lights reflected by aerosols towards the cam- tion image J so that the pristine image I = I^ − J can be ^ era. The formation of light-polluted image I is schemat- restored. To simplify the problem, we assume that for each ically depicted in Fig. 2. Although precise recovery of I color band λ, λ 2 fR; G; Bg, the strength of artificial light- ^ or equivalently J from I in terms of atmosphere science ing has a uniform distribution on earth surface, with a con- is very difficult, we aim to develop a practical method that stant radiance Aλ (a restriction to be removed in the next can neutralize light pollution and approximate I in percep- section). tual sense. To this end, we derive an approximate physical Denote by Eλ(x; y; z) the pollution light irradiance of model for the light pollution effect J. color band λ at spatial location (x; y; z). To keep the im- The scattering of ground artificial lights by aerosols is age and world coordinates consistent, we let the y axis rep- the main cause of light pollution. The exact modeling of resent the altitude. If the pollution lighting has uniform light pollution is highly complex, if not impossible, as the strength and constant color everywhere on ground surface, scattering effects depend on the types, orientations, sizes, then Eλ (x; y0; z) can be considered a constant for any and distributions of aerosols permeating the atmosphere, as 0 given altitude y0 and wavelength λ0. Therefore, the irradi- well as wavelengths, polarization states, and directions of ance function Eλ(x; y; z) of artificial lighting is reduced to the ground lights [13, 19, 21, 22]. We simplify the develop- a univariate function Eλ(y) that depends on altitude only, ment of light pollution model by assuming homogeneous at- λ 2 fR; G; Bg. Using the light attenuation model Eq(3), mosphere, namely, aerosols have uniform density and they we compute Eλ(y) in the atmosphere by integrating the in- scatter lights isotropically. fluxes of ground artificial lights that reach a point of altitude Practical light scattering models seemed to follow the y, as illustrated in Fig. 3, and obtain the radiance of the light work of Narasimhan and Nayar [21]. A light gets attenu- pollution at the atmosphere point ated as it travels. Due to aerosol scattering, a fraction of p 2 2 light flux is removed from the incident beam, and the re- Z 1 A e−βλ x +y E (y) = λ 2πx dx: (4) maining flux arrived at the destination point is the attenu- λ 2 2 0 x + y ated irradiance given by Bouguer’s exponential law [1], With a change of variable x = pl2 − y2, Eq(4) can be −βλd E(d; λ) = E0(λ)e ; (2) rewritten as Z 1 −βλl where E0 is the radiance of the light source prior to at- e Eλ(y) = 2πAλ dl; (5) tenuation, d is the distance from the source to destination y l Figure 3: The pollution light irradiance in atmosphere is an integration of the energy that ground artificial lights radiate. Figure 5: The perspective projection model of a camera. A pixel (x; y) in image J corresponds to a beam of pollution lights towards the camera. the light pathway, we obtain the pixel value of pollution image, ! Z L y + h −βλτ Jλ(x; y) = Eλ τ · βλe dτ; p 2 2 2 0 f + x + y λ 2 fR; G; Bg; Figure 4: The artificial light irradiance Eλ(y) vs. the alti- tude y, for different levels of air purity. 2:8 × 10−5 repre- (8) sents aerosol free, 10−4 represents slightly haze, and 10−3 represents haze. where L is the path length between the sensor pixel (x; y) and the scene point. For pixels in the sky, L is set to infin- where l is the distance between the ground light source and ity.