Blind Optical Aberration Correction by Exploring Geometric and Visual Priors
Total Page:16
File Type:pdf, Size:1020Kb
Blind Optical Aberration Correction by Exploring Geometric and Visual Priors Tao Yue1 Jinli Suo1 Jue Wang2 Xun Cao3 Qionghai Dai1 1Department of Automation, Tsinghua University 2Adobe Research 3School of Electronic Science and Engineering, Nanjing University Abstract have also proposed various complex hardware solutions for compensating optical aberration [1,9,10,15,17,18], but the Optical aberration widely exists in optical imaging sys- requirement of using special devices limits their application tems, especially in consumer-level cameras. In contrast range, especially for consumer applications. to previous solutions using hardware compensation or pre- Given the limitations of hardware solutions, computa- calibration, we propose a computational approach for blind tionally removing optical aberration has attracted strong aberration removal from a single image, by exploring vari- interests and has been exensively studied in the past ous geometric and visual priors. The global rotational sym- decade [3–5, 12, 13]. However, most of existing aberra- metry allows us to transform the non-uniform degeneration tion correction methods [3, 5, 12] are non-blind, i.e. the into several uniform ones by the proposed radial splitting Point Spread Functions (PSFs) of the aberration need to be and warping technique. Locally, two types of symmetry calibrated beforehand. Kee et al. [5] propose a parameter- constraints, i.e. central symmetry and reflection symmetry ized model to describe the nonuniform PSFs of a zoom lens are defined as geometric priors in central and surround- at different focal lengths, so that only several calibration ing regions, respectively. Furthermore, by investigating the measurements are required to be taken. Schuler et al. [12] visual artifacts of aberration degenerated images captured calibrate the nonuniform aberration PSFs using a grid of by consumer-level cameras, the non-uniform distribution of point light sources and apply the Efficient Filter Flow (EFF) sharpness across color channels and the image lattice is method to perform efficient non-stationary deconvolution. exploited as visual priors, resulting in a novel strategy to Heide et al. [3] propose to reconstruct high quality images utilize the guidance from the sharpest channel and local im- from aberration corrupted inputs captured by simple lenses, age regions to improve the overall performance and robust- with the PSFs calibrated from the degenerated images of ness. Extensive evaluation on both real and synthetic data several random patterns. Instead of a fully pre-calibration suggests that the proposed method outperforms the state-of- of the PSFs, Shih et al. [14] only measure the PSF at a the-art techniques. single depth, and then simulate the lens and trace the light rays to derive the whole PSF map computationally. The main advantage of these non-blind methods is that once the 1. Introduction camera is well calibrated, the model can be applied to any input image given the optical aberrations are scene inde- Optical aberration is one of the most common sources of pendent [20]. However the required calibration procedures image degeneration that affects almost all lens-based imag- usually demand expertise and special devices, thus largely ing systems. High-end lens are carefully designed to reduce limit their practical applications. optical aberration. For example, the elements of the com- For blind aberration removal, Joshi et al. [4] propose pound lens are made of different materials to compensate a PSF estimation method by predicting sharp edges from the imperfect refractive indices to reduce chromatic aberra- blurry ones. Rahbar and Faez [11] use the Zernike model to tion. The surfaces of the singlet lenses are of different cur- describe the optical aberration and estimate the Zernike co- vatures to compensate the imperfect focusing of the spheri- efficients using Polyspectral analysis. However, they only cal lenses for educing the spherical, astigmatism and coma deal with a single channel in this work and can not han- aberration. In general, better lens with less optical aberra- dle chromatic aberration. Recently, Schuler et al. [13] pro- tion are usually larger and cost much more to fabricate, thus pose an orthonormal Efficient Filter Flow (EFF) basis based may not applicable in consumer-level cameras. Researchers method which is built upon a series of assumptions. This 1 Optical path on y-z meridian plane Chromatic Spherical Coma Object points Optical lens CCD plane Optical principle for aberrations (a) (b) Figure 1. The diagram of optical principle of (a) optical aberrations and (b) the resulting PSFs. approach works well when the assumptions hold, but may cially for the seriously deteriorated images. fail in other cases, e.g., images with large spherical aberra- Compared with Schuler et al. [13]’s method, the pro- tion, as we will show in our experiments. Tang and Ku- posed algorithm differs in three main aspects: (1) it can tulakos [16] prove that the blind recovery of both optical handle more types of aberrations across the image plane, aberration and image depth from defocus blurred and aber- especially the spherical aberration in the center region and ration corrupted images is possible, and they use the Seidel coma in the surrounding regions. (2) Benefiting from the model to approximate the aberration and simplify the prob- sharp-to-blur strategy, the proposed method achieves better lem. In general, blind aberration removal is more practi- performance and robustness, especially for the chromatic cal than non-blind compensation, but technically more chal- aberration. (3) The radial splitting and warping framework lenging. simplifies the problem and gives rise to a great potential in The goal of this work is to provide a better blind aberra- terms of computational efficiency. tion correction method that achieves higher quality and bet- ter robustness/flexibility than previous approaches. Specif- 2. Optics-inspired Geometric Priors of PSF ically, we first investigate the principle of optics and derive the geometric priors of the PSFs, including both the global In this section, we analyze the principle of optics to de- rotational symmetry and the local reflectional/central sym- rive the geometric priors of the aberration PSFs. Fig. 1(a) metry. According to the global rotational symmetry, we shows the light paths that cause the spherical, coma and propose a radial splitting and warping framework to trans- chromatic aberrations. Generally, lens-based imaging sys- form the non-uniform aberrations in the concentric radial tems are axisymmetric with respect to the optical axis, thus regions into spatially uniform blurs. This transform largely the light paths of a well manufactured consumer-level cam- increases the efficiency and robustness of the algorithm, and era lens are also axis-symmetric, i.e., the off-axis points allows us to incorporate mature techniques that are proven with the same distance to the optical axis have the same to be effective in uniform deblurring. In addition, we pro- optical transform function (OTF). In addition, the OTF at a pose a more thorough local symmetry prior that includes specific point should also be locally symmetric with respect both the reflection symmetry used in [13] and a central sym- to the line connecting it to the projection center, which we metry constraint. This enables us to handle both the off-axis assume is the same as the image center. Neglecting the axis- PSFs (astigmatism and coma) and near-axis PSFs (spherical asymmetric error due to imperfect lens manufacturing, all and chromatic). types of lens aberrations, e.g. spherical aberration, coma, Beside the geometric priors, we also introduces new vi- and chromatic aberration, would cause PSFs that are glob- sual priors. Inspired by the observation that in an optically ally rotational symmetric and locally central/reflective sym- aberrated image, sharpness varies spatially as well as across metric. color channel, we propose to conduct both kernel estimation To describe these two geometric priors of PSFs, we pro- and deconvolution in a sharp-to-blur order. Specifically, we pose a radial splitting and warping framework and a unified restore the sharpest channel and local region first, and then symmetry constraint, as described below. use the result to guide the processing of other regions and channels. The guidance is implemented via two visual pri- Radial Splitting and Warping Framework The global ors of the degenerated images: cross-channel constraint on rotational symmetry has been incorporated into a series of the latent sharp image and the smoothly-varying constraint human designed basises in Schuler et al. [13]. In this work, on the blur kernels. This strategy and related visual priors we propose a radial splitting and warping framework that significantly boost the performance of our method, espe- is more flexible than the EFF-basis-based method. Consid- r = 0~125 pixels Specifically, in our radial split framework, the PSF of the center patch (a solid circle witgh no warping) is constrained r = 125~250 pixels to be central symmetric, while the reflection symmetry con- straint is applied for the PSFs in the outer rings. Next, we formulate these two symmetry constraints in r = 500~750 pixels a unified mathematical term. The basic idea is to divide the PSFs into sets according to the specific type of symme- try, and then minimize the summation of intra-set variances. Original image Ring splitting Warping Mathematically it is defined as: Figure 2. The diagram of the proposed concentric ringwise split- k XX 1 X 2 ting and warping. Csym(K)=λs (K(i)− K(m)) ; (1) jSjj j i2Sj m2Sj ering that the pixels in a narrow ring region have roughly where K denotes the PSF, j indices the sets Sj whose size is the same PSF with different rotations, if we rotate the local jSjj, and i is the index of the elements in a set. The weight patch of all pixels properly, the PSFs become uniform. Intu- of symmetry constraint λs is set to be 20 in this paper.