Correcting the Chromatic Aberration in Barrel Distortion of Endoscopic Images Y. M. Harry Ng Department of Automation and Computer-Aided Engineering The Chinese University of Hong Kong [email protected] and C. P. Kwong Department of Automation and Computer-Aided Engineering The Chinese University of Hong Kong [email protected] ABSTRACT on least square estimation. Several researchers have proposed various mathematical models of image Modern endoscopes offer physicians a wide-angle field of distortion and techniques to find the model parameters for view (FOV) for minimally invasive therapies. However, distortion correction. For instance, Hideaki et al. [1] have the high level of barrel distortion may prevent accurate derived a moment matrix from a set of image points and perception of image. Fortunately, this kind of distortion proposed to straighten distorted grid lines in the image may be corrected by digital image processing. In this based on the smallest characteristic root of this matrix. paper we investigate the chromatic aberrations in the Asari et al. [2] have proposed a technique based on least barrel distortion of endoscopic images. In the past, squares estimation to find the coefficients of a correction chromatic aberration in endoscopes is corrected by polynomial, to which Helferty et al. [3] later provided an achromatic lenses or active lens control. In contrast, we improved method. Specifically they proposed a technique take a computational approach by modifying the concept which finds solutions in which all lines are parallel. of image warping and the existing barrel distortion Unlike previous approaches, both horizontal and vertical correction algorithm to tackle the chromatic aberration line fits are required to find the optimal distortion center problem. In addition, an error function for the and hence the overall solution. Nevertheless, the above determination of the level of centroid coincidence is methods concern only the geometric barrel distortion. In proposed. Simulation and experimental results confirm our case, we are interested in reducing the chromatic the effectiveness of our method. aberration in the barrel distortion of endoscopic images. Keywords: Barrel Distortion, Chromatic Aberration, For the correction of chromatic aberration, an achromatic Distortion Correction, Endoscopy, Centroid Estimation doublet consisting of two thin lenses that are separated or and Expansion Coefficients. in contact so that its focal length is independent of wavelength is used [4][5]. Normally one lens is 1. INTRODUCTION converging and the other is diverging, and they are often cemented together. As it is not possible to fully correct all Minimally Invasive Therapy is increasingly popular aberrations, combining two or more lenses together for because of the use of natural or artificial orifices of the the correction is required. High-quality lenses used in body for surgical operations, which minimizes the cameras, microscopes, and other devices are compound destruction of healthy tissues and organs. Electronic lenses consisting of many simple lenses. A typical endoscopy employing miniature CCD cameras plays an high-quality camera lens may contain six or more lenses. important role in the diagnostic and therapeutic On the other hand, Willson et al. [6] proposed active lens operations of Minimally Invasive Therapy. However, the control to correct the chromatic aberration. In the modern high level of barrel distortion introduces nonlinear endoscope, compound achromatic lenses for correcting changes in the image: areas near the distortion center are chromatic aberration are commonly used. compressed less and areas farther away from the center In this paper, we investigate the chromatic aberration in are compressed more. Because of this phenomenon, the the barrel distortion of endoscope. J. Garcia et al. [7] have outer areas of the image look significantly smaller than extracted three RGB color channels of an image for the the actual size. Fortunately, this kind of distortion may be measurement of the defocusing from the continuous light corrected by calculating the correction parameters based spectrum. Because of the depth of field, we assume the SYSTEMICS, CYBERNETICS AND INFORMATICS VOLUME 1 - NUMBER 2 81 images are focused already and the lateral chromatic colors from violet to red. Light with wavelength shorter aberration can be regarded as difference in magnification than 400 nm is called ultraviolet (UV), and light with for different colors. Indeed, the chromatic correction can wavelength greater than 750 nm is called infrared (IR). A be performed either using a theoretical model of prism separates white light into rainbow of colors. This aberration formation or using experimentally measured happens because the index of refraction of a material aberration values. However, it is practically impossible to depends on the wavelength. With reference to the concept describe contributions of all parts of a particular optical of correcting chromatic aberration using image warping system. Therefore, the use of experimentally measured from Boult et al. [8], the geometric distortions must be aberration values is a much better choice. Thus, we take computed and the red and green images must be fitted to three monochromatic light samples, which are regarded a cubic spline. Since the light is a continuous spectrum, as three RGB color channels. Then, we take a we can not deal only with three separate channels. In computational approach by modifying the concept of order to replace the achromatic lenses by computational image warping [8] and the existing barrel distortion correction, we need to take as many light samples as we correction algorithm by Helferty et al. [3] to tackle the can. In the meantime, we develop an error function which chromatic aberration problem. Once the correction is used to determine how good the fitting of a cubic polynomial coefficients are found, the correction of the spline in the chromatic aberration in barrel distortion of chromatic aberration in barrel distortion of endoscopic endoscopic images is. We assume that the tested white images can be carried out. As the polynomial expansion light source only consists of three monochromatic light coefficients are not changed with the distance from the sources and they are belonged to the three color channels. object to the camera lens, the expansion coefficients Thus, the mappings from the distortion space to the obtained at a certain distance can be used to correct the correction space for the tested white light source are, image captured at any distance within the depth of field range [9]. In addition, an error function for the N n , θ = θ ' determination of the level of centroid coincidence is rR = ∑anRr'R R R proposed. Finally, we use simulation and experimental n=1 results to confirm the effectiveness of our method. N n , θ = θ' rG = ∑anGr'G G G 2. THERORETICAL MODEL OF n=1 CORRECTION N n , θ = θ ' (2) There have been many studies on the computational rB = ∑anBr'B B B correction of barrel distortion. In this paper, we mainly n=1 focus on the method proposed by Helferty et al. [3], of where , and are the set of correction which the correction of the mapping from distortion anR anG anB space to correction space for black and white images is, expansion coefficients in red, green and blue color, respectively, and r' and r are the radii from the N distortion center to the point in the distortion space and n , θ = θ ' (1) ρ = ∑ an ρ' the correction space, respectively with its corresponding n=1 color. θ ' and θ are the angles of the point in the distortion space and the correction space, respectively where a is the set of correction expansion coefficients n with its corresponding color. The goal of the processing is of an endoscope, ρ ' and ρ are the radius from the to calculate the coefficients of a1R ,...anR , a1G ,...anG , distortion center to a point in the distortion space and the a ,...a . correction space, respectively, and θ ' and θ are the 1B nB angles of the point in the distortion space and the correction space, respectively. The basic experimental Although we have found the coefficients for three colors, procedure, as suggested by Helferty et al., is to compute the corrected images may not be coincided with each the correction expansion coefficients from an endoscopic other. In order to compensate the dispersion of the three video image of an image consisting of an equally spaced colors, we make fine adjustments of the correction array of black dots on a white background. Once this polynomial coefficients. Thus, an error function is now correction is computed, it can be applied to live being proposed. As the green color centroid is located in endoscopic video in real time. the middle of red and blue color centorids, we choose the green color centroid and the green color correction An important property of visible light is its color, which polynomial coefficient as the desired control point. Then, is related to the wavelengths or frequencies of light. we can finely adjust the correction polynomial Visible light sensitive to human eyes falls in the coefficients by the following proposed method. First, we wavelength range of about 400 nm to 750 nm. This is compute the distance of the farthest dot centroid in 45 known as the visible spectrum and represents different degree from the center point of the image as, 82 SYSTEMICS, CYBERNETICS AND INFORMATICS VOLUME 1 - NUMBER 2 1 N xc = x (5) c 2 c 2 (3) ij ∑ ijn D = (x ) + ( y ) N n where D is the distance of the farthest dot centroid in c where xij is the value in the x-direction of the centroid 45 degree from the center point of the image, c is the x of dot_ij (row i and column j), x is the value in the value of the dot centroid in the x-direction and yc is the ijn x-direction of the dot_ij elements and N is the number value of the dot centroid in the y-direction.