Homography Observer Design on Special Linear Group SL(3) with Application to Optical Flow Estimation Ninad Manerikar, Minh-Duc Hua, Tarek Hamel
Total Page:16
File Type:pdf, Size:1020Kb
Homography Observer Design on Special Linear Group SL(3) with Application to Optical Flow Estimation Ninad Manerikar, Minh-Duc Hua, Tarek Hamel To cite this version: Ninad Manerikar, Minh-Duc Hua, Tarek Hamel. Homography Observer Design on Special Linear Group SL(3) with Application to Optical Flow Estimation. European Control Conference (ECC), Jun 2018, Limassol, Cyprus. hal-01878702 HAL Id: hal-01878702 https://hal.archives-ouvertes.fr/hal-01878702 Submitted on 21 Sep 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Homography Observer Design on Special Linear Group SL(3) with Application to Optical Flow Estimation Ninad Manerikar, Minh-Duc Hua, Tarek Hamel Abstract— In this paper we present a novel approach for based on the group structure of the set of homographies, homography observer design on the Special Linear group the Special Linear group SL(3). This nonlinear observer SL(3). This proposed approach aims at computing online, the enhances the overall homography estimate between any two optical flow estimation extracted from continuous homography obtained by capturing video sequences from aerial vehicles given images by using the velocity information in order equipped with a monocular camera. The novelty of the paper to interpolate across a sequence of images. However, this lies in the linearization approach undertaken to linearize a observer still requires the pre-computation of the individual nonlinear observer on SL(3). Experimental results have been image homographies, which are then smoothed using filter- presented to show the performance and robustness of the ing techniques, thus making this approach computationally proposed approach. expensive. Also one of the drawbacks of this approach is that I. INTRODUCTION in order to obtain improved homography estimates, each pair of images needs to have a sufficient amount of data features Originated from computer vision, the homography repre- for the computation of homography. In another work [7] by sents a mapping that relates two views of the same planar some of the authors of the paper, the question of deriving an scene. It encodes the camera’s pose (i.e. position and orienta- observer that directly uses point correspondences from an tion), the distance between the camera and the scene and the image sequence without requiring the prior reconstruction scene’s normal vector in a single matrix. The homography of the individual homographies has been addressed. This has played an important role in various computer vision approach is not only computationally more efficient but also and robotics applications, where the scenarios involve man- works for scenarios where there is lack of data features for made environments that are composed of planar surfaces [1], the reconstruction of the homography matrix. [2]. It is also suitable for applications where the camera is In the present paper, we address the same problem as sufficiently far from the viewed scene, such as the situations [7] but with a different approach that directly exploits the when ground images are taken from an aerial drone. basis of the Lie algebra of the group SL(3) for observer The homography estimation problem has been extensively design. The simplicity of the resulting observer makes it studied in the past [3], [4], [9]. Most existing works consider suitable for real-time implementation. The proposed observer the homography as an incidental variable and tend to solve a is also ideal for the estimation of the so-called “continuous system of algebraic constraints on the frame-by-frame basis homography” and the optical flow by exploiting the homog- [9]. The quality of the homography estimates is strongly raphy estimated from every two consecutive images obtained dependant on the algorithm used as well as the nature of from a combined Camera-IMU (Inertial Measurement Unit) the image features used for estimation. Image features for system. homography estimation are typically geometric (points, lines, The paper is structured into six sections including the conics, etc.) or texture. For a well textured scene, state- introduction and the concluding sections. Section II gives of-the-art methods can provide high quality homography a brief description of the notation and the math related to estimates at the cost of high computational complexity homography. In section III a linear approach for observer (see [4]). For a scene with poor texture, while significant design on SL(3) is proposed using point correspondences computational effort is still required, the poor quality of and the knowledge of the group velocity. In section IV homography estimates is often an important issue. This is the computation of the optical flow estimate extracted from particularly the case in varying lighting conditions and in continuous homography is presented. Experimental results presence of specular reflections or moving shadows, where supporting the proposed approach are presented in section photogrammetric error criteria used in texture-based methods V. Finally, some concluding remarks and future works are become ill-posed. Feature-based methods of image-to-image provided in section VI. homography estimation are robust to these issues as long as good features and good matching are obtained. They have II. NOTATION AND THEORETICAL BACKGROUND been the mainstay of robust computer vision in the past. A. Mathematic background In a previous work [5], [6] by some authors of this paper, a The special linear group SL(3) and its Lie algebra sl(3) nonlinear observer for homography estimation was proposed are defined as 3×3 N. Manerikar, M.-D. Hua, T. Hamel are with I3S, SL(3) := H 2 R j det(H) = 1 Universite´ Coteˆ d’Azur, CNRS, Sophia Antipolis, France, 3×3 manerika(hua; hamel)@i3s:unice:fr. sl(3) := U 2 R j tr(U) = 0 Since the Lie algebra sl(3) is of dimension 8, it can be C. Homography kinematics spanned by 8 generators so that for any ∆ 2 sl(3) there Let Ω and V denote the rigid body angular velocity and 8 exists a unique vector δ 2 R such that linear velocity of the current frame fAg w.r.t the reference 8 frame fA˚g, both expressed in fAg. Then the rigid body X ∆ = δiBi (1) kinematics are R_ = RΩ i=1 × (2) ξ_ = RV where the basis of sl(3) are chosen as follows: with [·] denoting the skew-symmetric matrix associated > > > × B1 = e1e2 ;B2 = e2e1 ;B3 = e2e3 with the cross product, i.e. u v = u × v; 8u; v 2 3. > > > × R B4 = e3e2 ;B5 = e3e1 ;B6 = e1e3 Let us consider that the stationary planar scene is viewed > 1 > 1 B7 = e1e1 − I;B8 = e2e2 − I by a camera attached to the rigid body with kinematics (2). 3 3 Let H 2 SL(3) denote the associated homography matrix. 3×3 with I the identity element of R and fe1; e2; e3g the Then, its kinematics are given by canonical basis of 3. R H_ = HU (3) B. Homographies where the group velocity U 2 sl(3) induced by the relative Let A˚ (resp. A) denote projective coordinates for the motion between the camera and the stationary planar scene image plane of a camera A˚(resp. A), and let fA˚g (resp. fAg) satisfies [5] denote its frame of reference. The position of the frame fAg V η> η>V with respect to fA˚g expressed in fA˚g is denoted by ξ 2 3. U = Ω + − I (4) R × d 3d The orientation of the frame fAg with respect to fA˚g is represented by a rotation matrix R 2 SO(3). Let d˚ (resp. The group velocity U given by (4) is often referred to as d) and ˚η 2 S2 (resp. η 2 S2) denote the distance from the “Continuous Homography” in the literature [8]. origin of fA˚g (resp. fAg) to the planar scene and the normal III. OBSERVER DESIGN ON SL(3) FOR HOMOGRAPHY vector pointing to the scene expressed in fA˚g (resp. fAg). ESTIMATION Hence the coordinates of the same point in the reference frame (P˚ 2 fA˚g) and in the current frame (P 2 fAg) can The equation of the proposed homography estimator can be expressed by the relation be expressed as a kinematic filter system on SL(3) as _ P˚ = RP + ξ H^ = HU^ + ∆H^ (5) If the intrinsic parameters of the camera are known meaning where the innovation term ∆ 2 (3) has to be designed in ~ ^ −1 that the camera is calibrated one can write1 order to drive the group error H := HH to identity, based on the assumption that we have a collection of n −1 ˚ ∼ −1 ∼ −1 H ˚yj 2 P = K ˚p; P = K p measurements yj = −1 2 S (j = 1; : : : ; n), with jH ˚yj j ˚y 2 S2 known and constant. Here y and ˚y represent where K is the camera calibration matrix and ˚p 2 A˚ (resp. j j j calibrated image points normalized onto the unit sphere and p 2 A) is the image of the point P˚ (resp. P ) when the can be computed as camera is aligned with frame fA˚g (resp. fAg) and can > −1 −1 be written in the form (u; v; 1) using the homogeneous K pj K ˚pj yj = −1 ; ˚yj = −1 coordinate representation for that 2D image point. The image jK pjj jK ˚pjj homography matrix that maps pixel coordinates from the e current frame to the reference frame is given by The output errors j are defined as ^ ~ > Hyj H˚yj ξη −1 ej := = (6) Him = γK R + K ^ ~ d jHyjj jH˚yjj where γ is a scale factor that, without loss of generality, can which thus can be viewed as the estimates of ˚yj.