A Geometric Correction Method Based on Pixel Spatial Transformation
Total Page:16
File Type:pdf, Size:1020Kb
2020 International Conference on Computer Intelligent Systems and Network Remote Control (CISNRC 2020) ISBN: 978-1-60595-683-1 A Geometric Correction Method Based on Pixel Spatial Transformation Xiaoye Zhang, Shaobin Li, Lei Chen ABSTRACT To achieve the non-planar projection geometric correction, this paper proposes a geometric correction method based on pixel spatial transformation for non-planar projection. The pixel spatial transformation relationship between the distorted image on the non-planar projection plane and the original projection image is derived, the pre-distorted image corresponding to the projection surface is obtained, and the optimal view position is determined to complete the geometric correction. The experimental results show that this method can achieve geometric correction of projective plane as cylinder plane and spherical column plane, and can present the effect of visual distortion free at the optimal view point. KEYWORDS Geometric Correction, Image Pre-distortion, Pixel Spatial Transformation. INTRODUCTION Projection display technology can significantly increase visual range, and projection scenes can be flexibly arranged according to actual needs. Therefore, projection display technology has been applied to all aspects of production, life and learning. With the rapid development of digital media technology, modern projection requirements are also gradually complicated, such as virtual reality presentation with a high sense of immersion and projection performance in various forms, which cannot be met by traditional plane projection, so non-planar projection technology arises at the right moment. Compared with traditional planar projection display technology, non-planar projection mainly includes color compensation of projected surface[1], geometric splicing and brightness fusion of multiple projectors[2], and geometric correction[3]. The geometric correction studied in this paper takes into account that in non-planar projection, the geometric structure of the projection surface will affect the visual perception of the original projected image on the projection surface, greatly affecting the projection effect. Existing geometric correction methods, such as Sajadi[4-5] proposed a geometric correction method suitable for cylinders. This method uses the camera to extract the edge of the cylindrical projection surface, and calculates the camera and projector parameters according to the structural characteristics of the cylindrical surface and the edge information iteratively. Steimle[6] proposed a flexible surface projection technology, which focuses on using surface fitting to obtain the functional expression __________________ Xiaoye Zhang, Shaobin Li, Lei Chen, School of Information and Communications Engineering Communication University of China Beijing, China 302 of the projected surface. A key step is to automatically obtain the 3d key point position of the projected surface through the depth camera, without manually setting markers or visible textures. Bermano[7] proposed a dynamic real-time facial enhancement system, which USES infrared cameras to shoot performers' faces, assigns different fusion features to specific facial features, and deforms the 3D motion capture BlendShape model and markers detected in 2D images to achieve accurate projection. The above methods mostly use the auxiliary equipment to complete the geometric correction, which greatly limits the usage scenario. Considering that the basic unit of image composition is pixel, and the distortion of the projected image is caused by the change of pixel position and size of the original projection image due to the structure of the projection surface, therefore it is an intuitive geometric correction method to directly correct the position of distorted pixels on the projection plane and correct them back to the ideal position. PRICIPLE OF GEOMETRIC CORRECTION Pixel Spatial Transformation In order to deduce the spatial transformation relationship of pixels intuitively, this paper takes the spherical projection surface shown in Figure 1 as an example to illustrate the relationship among distorted image, original projection image and pre-distorted image on the projection surface. As shown in Figure 1, it is a spherical projection system. In the figure, the projector is at O point, following the perspective projection principle, and the vertex of the apparent cone is O point. The surface KLMNS is the spherical projection surface, and the plane KLMN is the virtual plane set before the projection to facilitate subsequent calculation. Figure 1. Spherical Projection System. When the projector at Point O projects the image, the projection line OAA' of pixel P passes through the virtual plane KLMN and the actual projection surface KLMNS successively, and the intersection point is A and A' respectively. The image formed by all pixel points on the virtual plane is not distorted, and there is only a scaling relationship between the projected image on the virtual plane and the original projection image. If the scaled image on the virtual plane is directly vertically mapped to the actual projection plane behind, the correct projected image will be obtained, that is, the vertical mapping point B of point A is the ideal position of pixel point P. 303 But in fact, after perspective projection, the pixel position is distorted, that is A' is the actual position of pixel P. Therefore, the geometric correction can be achieved by correcting the actual position A' of pixel P to the ideal position B of pixel P. However, both points A' and B are on the sphere, which means the two points are offset in the three directions of XYZ, so the quantitative calculation is complicated. According to the perspective projection principle, point B also has a corresponding expected position point C on the virtual plane KLMN, and the corresponding backward projection line is OCB. Therefore, the position relation of A' and B is converted to the corresponding A and C of these two points on the virtual plane, and then the transformation function is calculated. In order to express the geometric transformation relationship more clearly, the OCB projection line was used to intercept the cross-section along the X-axis, as shown in Figure 2. And the overall top view with projection line OCB, as shown in Figure 3: Figure 2. Projection line OCB section. Figure 3. Spherical projection system. According to Figure 2, triangle similarity can be obtained as follows: According to Figure 3, the line segment OCB (OC'B') is the OBB' plane in Figure 2. Also according to the triangle similarity, we can get: So there is: When the coordinate value is substituted, it is: 304 Point A is the vertical mapping point of point B, so the X and Y coordinates of point A and B are the same. B’’ and C’’ are mapping point of point O, so the Y coordinates of these 3 points are the same. B’, C’ and O are on the same horizontal plane, so the X coordinates of these 3 points are the same. Line BB’’ is parallel to the XY plane, which means perpendicular to the Z axis, so the Z coordinates of B and B’’ are the same. Moreover, point B is on the spherical projection surface, so if we know that the equation of the sphere is function f, then the Z coordinates of B’’ can be represented by f and coordinates of point A. Denotes d as the distance between point O and the virtual plane, i.e the Z coordinates of point O. So we can get: Then we can get the relationship of point A and C: Pixel Spatial Transformation After the position of each pixel on the pre-distorted image is determined, the pixel gray value (RGB chromaticity value) is also considered. Pre-distortion directly results in the change of image size, squeezing or stretching of pixels, and obvious change of pixel position. As shown in Figure 4. is the original rectangle and the result of direct pre-distortion. Figure 4. Initial pre-distored effect based on cylinder. In the pre-distorted image obtained, the pixel position does not necessarily fall at the integer pixel. Moreover, the reference point of pixel transformation during image pre-distortion is determined based on the virtual plane closer to the projector, so the effective pixel area is also reduced. Therefore, it is necessary to fill the entire effective pixel area with pixel interpolation and restore it to the size of the original image, so that the pre-distorted image has the same gray value information and size consistent with the original image. Traditional image interpolation is mainly used for image scaling[8]. The most adjacent point interpolation, bilinear interpolation and bi-cubic interpolation are commonly used. The difference is that the number of points required is different. 305 There are two interpolation operations in this paper, one is to obtain the pixel gray value of the pre-distorted image, and the other is to restore the size of the original pre-distorted image. Since the shape of the initial pre-distortion image is different from the original image, it is no longer suitable for the traditional interpolation method, so the two interpolation operations are slightly different. Bilinear interpolation method is used in both two interpolation methods in this paper, as shown in Figure 5. Figure 5. Bilinear interpolation. For point on the original image, its pre-distorted coordinates is . Due to point Q coordinates are not necessarily an integer, and the effective pixel region of the original image becomes smaller, so there may be multiple pixel points corresponding to the pixel points in the original image in an integer pixel grid of the pre-distorted image, so in order to establish contact with integer coordinates, round coordinates of Q down, respectively for and , therefore point Q is located in a rectangular area, as shown in Figure 6: Figure 6. Location of point Q. The distance between point Q and the integer coordinate in both directions is used as the weight of adjacent points. Experimental Results The experiment was carried out in Unity.