Fresnel Lens Imaging with Post-Capture Image Processing Artem Roman Vladimir Maksim Sergey Yuriy Nikonorov Skidanov Fursov Petrov Bibikov Yuzifovich Samara State Aerospace University Image Processing Systems Institute of RAS 34, Moskovskoye Shosse, Samara, Russia, 443086 151, Molodogvardeyskaya str., Samara, Russia, 443001 artniko romans fursov max.vit.petrov bibikov.sergei yuriyvyuzifovich @gmail.com @smr.ru @ssau.ru @gmail.com @gmail.com @gmail.com example, the weight of a typical 300mm lens exceeds 1 Abstract kg, while an equivalent Fresnel lens can weigh less than 50 grams. It is also easy to fabricate Fresnel lenses with This paper describes a unified approach to correct an aspheric wavefront. optical distortions in images formed by a Fresnel lens with computational post-processing that opens up new opportunities to use Fresnel lenses in lightweight and inexpensive computer vision devices. Traditional methods of aberration correction do not address artifacts introduced by a Fresnel lens in a systematic way and thus fail to deliver image quality acceptable for general- purpose color imaging. In our approach, the image is restored using three steps: first, by deblurring the base color channel, then by sharpening other two channels, and finally by applying color correction. Deblurring and sharpening remove significant chromatic aberration and Figure 1: An image formed by a binary Fresnel lens (left); an are similar to the restoration technique used for images image formed by a four-step Fresnel lens approximation with formed by simple refraction lenses. Color correction computational correction. stage removes strong color shift caused by energy redistribution between diffraction orders of Fresnel lens. However, these advantages come at a cost: the resulting This post-capture processing was tested on real images point spread function depends on the light wavelength and formed by a four-step approximation of the Fresnel lens has multiple distortions such as moiré and chromatic halo. manufactured in our optics laboratory. As a result, Fresnel lenses are typically used as optical collimators or concentrators and not as imaging lens [3]. Other common applications include chromatic aberration 1. Introduction compensation using doublets with refractive lenses [4], and as an element in X-Ray microscopes [5, 6]. Modern camera lenses have become very complex. Fresnel lenses have stronger chromatic aberration than They typically consist of a dozen or more elements to simple refractive lenses do. One of the color channels (in remove optical aberrations [1]. Recently, simple lenses this paper we use the green channel) has less blurring and with one or two optical elements were proposed [2]. can be used as a reference channel for the correction of These lenses are similar to lenses used hundreds years the other two channels. In this paper, we also discuss light ago, where various aberrations, especially chromatic energy redistribution between diffraction orders that cause aberrations, are addressed in post-processing. Recently strong chromatic shift. proposed post-processing techniques can significantly Chromatic aberrations in distorted images can be reduce the distortions [2]. With a computational algorithmically corrected based on the blind or semi-blind correction, simple lenses can make optical systems deconvolution [2], or with a sharpening based on contour inexpensive and lightweight. analysis in different color channels [7]. In [8], a combined Fresnel lenses can be used as imaging lenses and offer technique is used. a further improvement in weight and linear size to quality Chromatic aberration model is derived as a ratio over refractive lenses. This advantage is especially generalization of optical system defocus model. pronounced for long focal lengths, where a single Fresnel Richardson [9] and Lucy [10] proposed iteration lens can replace a complex set of refractive lenses. For deconvolution method for optical defocus compensation with an aperture of up to f/2. Diffraction efficiency of in astronomical observations. In recent years, a modified these lenses is close to 80% for the base wavelength. approach is used to correct chromatic aberration [8, 11, Chromatic distortions for this lens exceed the 12]. distortions produced by refraction lens. Focus distance of We use both deblur and sharpening to remove the Fresnel lens depends on the wavelength of incident chromatic aberrations from images formed with Fresnel light. Phase zone plate calculated for the base wavelength lenses. Then we apply color correction based on λ0, and maintaining focal length f0 for this wavelength technique proposed in [13] to remove strong chromatic will focus light of wavelength λ on focal distance f, shift. defined by the following expression: Fresnel lens is an approximation of lens surface as λ ff= 0 . (1) shown in Figure 2. We use a four-step approximation of 0 λ the Fresnel lens created by consecutive etching with A Fresnel lens typically adds strong chromatic different binary masks [14]. Finally, we show the results distortion in non-monochromatic light. For wavelength of the computational correction applied to real images further away from λ0, diffraction efficiency of the zero formed by Fresnel lens to prove that this alternative optics order decreases. The light focused in the zero order can be used in imaging applications. A comparison creates an additional chromatic highlight. This highlight between image captured by binary Fresnel lens and image becomes stronger as the wavelength deviates from λ0. captured by our four-step Fresnel lens approximation with Diffraction efficiency of zero order can be expressed as: computational post-capture processing is shown in π Figure 1. τ=τ2 ⎡() − ⎤ 0 cos⎢ nh 1⎥ , (2) ⎣ λ⎦ where τ is transmittance coefficient in the zero order direction, τ0 total lens transmittance coefficient, h - height of Fresnel lens microrelief, n – refraction index. We will call the color highlights caused by the energy focused in non-working diffraction orders as chromatic shift, in addition to chromatic aberration. Chromatic aberration leads to color fringe along the edges and color shift distort colors of plane colored parts of the image. Figure 2: Conceptual illustration of collapsing aspheric We use first diffraction order as work order. We made refraction lens into Fresnel lens. numerical simulations of focusing process of binary Fresnel lens. As shown in Figure 4, focusing distance shifts for wavelengths deviated off the base wavelength. 2. Image capturing through Fresnel lens This shift leads to axial chromatic aberration. When a certain wavelength is out of focus, the energy passes to A Fresnel lens used in this work was fabricated using different diffraction orders, causing additional chromatic photolithography with spatial resolution of 800nm [15]. shift. Three steps of thermo-chemical thin chrome films writing where used before plasma-chemical etching. An image of the central part of the manufactured Fresnel lens, obtained with white light interferometer (New View Zygo 5000), is shown in Figure 3. Figure 4: Intensity distribution on the optical axis of binary Fresnel lens (blue line for λ=400 nm, green – for 550 nm and red for 700 nm). In the next section, we introduce a distortion model as the foundation for the step-by-step computational correction of chromatic aberrations and chromatic shift. Figure 3: Central part of produced Fresnel lens. This fabrication technique can produce Fresnel lenses 3. Image correction for Fresnel lenses deblurred green channel, the sharpest one, by a deconvolution: DDB=⊗−1, Chromatic aberration in refraction lenses is described ppGGG()xB() () x. (6) by the general defocus model [2]. In this model, the blur − Here operation B 1 ⊗ is a deconvolution for the kernel, or point spread function (PSF), is supposed to be k D linear, at least in local spatial area, as shown in: chromatic deblurring, with an intermediate image pxG () B =⊗0 + ppRGB()xB RGB () x n, (3) as a result. Then we apply a sharpening to red and blue channels where p B ()x is the one of the red, green, or blue color RGB using deblurred green channel as a base: 0 DDBD, channels of the blurred image, and pRGB ()x is the = pSppRB()xxx ( RB (), G ()). (7) corresponding color channel of the underlying sharp Finally, we apply color correction to the obtained image, B is a blur kernel, or PSF, n is additive image image: ∈Ζ2 noise, x + is a point in image spatial domain. = DD pFppRGB()xxx ( RB (), G ()). (8) Paper [16] shows that the lens PSF varies substantially Fp((),())DDxx p is a color correction transformation. being a function of aperture, focal length, focusing RB G distance, and illuminant spectrum. So, the assumption that Similar to sharpening, we use information available in the the blur kernel B in (3) is constant is not accurate enough, green channel to correct color shift in red and blue especially for Fresnel lenses with strong chromatic channels. aberration. Combining the above steps, we propose the following For a strong aberration, kernel B is space-varying. technique based on model (4) - (5): There are two types of distortions in the image: space- 1) the chromatic deblurring (6) of the green channel varying chromatic blur along the edges and color shift in based on the deconvolution, described in Section III); the regions with plain colors. Therefore, we use the 2) the chromatic sharpening (7) of the blue and red following modification of (3) to handle these distortions: channels using the contours analysis (this approach is described in Section IV); ppDB, ()xB=⊗ D () x + n, (4) RGB RGB RGB 3) the color correction (8) to remove color shift, which D = 0 pDpRGB(xx ) RGB ( RGB ( )). (5) is described in Section V. DB, These three steps of post-capturing correction are Here pRGB ()x are color channels of the image captured 0 shown in Figure 7. through Fresnel lens; DpRGB(()) RGB x is a component characterizing the color shift, caused by energy D,B D D pG Deblur pG Sharp pRGB Color pRGB redistribution between diffraction orders.