Mathematical Principles in Vision and Graphics: Rigid Transformations Ass.Prof. Friedrich Fraundorfer SS 2018 Learning goals . Understand the problems of dealing with rotations . Understand how to represent rotations . Understand the terms SO(3) etc. Understand the use of the tangent space . Understand Euler angles, Axis-Angle, and quaternions . Understand how to interpolate, filter and optimize rotations 2 Outline . Rigid transformations . Problems with rotation matrices . Properties of rotation matrices . Matrix groups SO(3), SE(3) . Manifolds . Tangent space . Skew-symmetric matrices . Exponential map . Euler angles, angle-axis, quaternions . Interpolation . Filtering . Optimization 3 Motivation: 3D Viewer Rigid transformations 푧 푝 푋퐶 푍 푋푤 푥 퐶 푦 푂 푊 푇 푅 푂 푌 푥 ∈ ℝ푛 푋 푋푐 = 푅푋푤 + 푇 T ∈ ℝ푛 . Coordinates are related by: 푋푐 푅 푇 푋푤 = 푛×푛 1 0 1 1 푅 ∈ ℝ . Rigid transformation belong to the matrix group SE(3) . What does this mean? 5 Properties of rotation matrices Rotation matrix: 푧 푍 3×3 푅 = 푟1, 푟2, 푟3 ∈ ℝ 푟3 푦 푟2 푇 푅 푅 = 퐼, det 푅 = +1 푂 푌 푟1 푋 푥 Coordinates are related by: 푋푐 = 푅푋푤 . Rotation matrices belong to the matrix group SO(3) . What does this mean? 6 Problems with rotation matrices . Optimization of rotations (bundle adjustment) 푓(푥푛) ▫ Newton’s method 푥푛+1 = 푥푛 − 푓′(푥푛) . Linear interpolation . Filtering and averaging ▫ E.g. averaging rotation from IMU or camera pose tracker for AR/VR glasses 7 Matrix groups . The set of all the nxn invertible matrices is a group w.r.t. the matrix multiplication: 퐺퐿(푛) = 푀 ∈ ℝ푛×푛|det(푀) ≠ 0 ,× General linear group . Reminder: A group is an algebraic structure consisting of a set of elements equipped with an operation that combines any two elements to form a third element. The operation satisfies four conditions called the group axioms, namely closure, associativity, identity and invertibility. Closure means for all a, b in G, the result of the operation, a • b, is also in G. 8 Matrix groups . The set of all the nxn orthogonal matrices is a group w.r.t. the matrix multiplication: 푂(푛) = 퐴 ∈ 퐺퐿(푛)|퐴−1 = 퐴푇 ,× Orthogonal group 퐴 ∈ 푂 푛 det 퐴 = ±1 9 Matrix groups . The set of all the nxn orthogonal matrices with determinant equal to +1 is a group w.r.t. the matrix multiplication: 푆푂(푛) = 퐴 ∈ 푂(푛)| det 퐴 = +1 ,× Special orthogonal group . SO(3) … group of orthogonal 3x3 matrices with det=+1 …. “rotation matrices” . R3 = R1*R2 … R3 is still an SO(3) element . R3 = R1+R2 … R3 is NOT an SO(3) element. Not a rotation matrix anymore. 10 Matrix groups . The set of all the rigid transformations in Rn is a group (not commutative) with the composition operation 퐹: ℝ푛 ℝ푛| F rigid ,∘ . The set is isomorphic to the special Euclidean group SE(n) . The mathematical properties of a “rigid transformation” are specified by the special Euclidean group SE(n) 푅 푇 푅푇 = 0 1 11 Matrix groups: Summary 푛×푛 GL(n) ℝ O(n) Vector space of all the nxn matrices SO(n) O(n)/SO(n) 퐺퐿(푛) = 푀 ∈ ℝ푛×푛|det(푀) ≠ 0 ,× General linear group 푂(푛) = 퐴 ∈ 퐺퐿(푛)|퐴−1 = 퐴푇 ,× Orthogonal group 푆푂(푛) = 퐴 ∈ 푂(푛)| det 퐴 = +1 ,× Special orthogonal group Set of orthogonal matrices which 푂(푛)/푆푂(푛) = 퐴 ∈ 푂(푛)| det 퐴 = −1 do not preserve orientation (not a group) GL(n), O(n), SO(n) and SE(n) are all smooth manifolds (e.g. surfaces, curves, solids immersed in some big vector space) 12 Special Euclidean group . The special Euclidean group is constructed by the Cartesian product (a composition operation) from SO(n)xRn. 푆퐸(푛) = 푆푂(푛) × ℝ푛,× 푀, 푡 × (푆, 푞) = (푀푆, 푀푞 + 푡) . The Cartesian product defines were the values of SO(n) and Rn (rotation and translation) go to form the transformation matrix 푅 푇 푅푇 = 0 1 3 푆퐸(3) = 푆푂(3) ℝ 0 1 13 Manifolds . Non-mathematical definition: Manifolds are shapes like curves, surfaces, and volumes in Euclidean space (i.e. a vector space) . A sphere is an example of a manifold of dimension 2. The matrix groups SO(3), SE(3) are M manifolds. 14 Shapes of SO(2) and SO(3) SO(2) … 1-manifold SO(3) … 3-manifold (3-sphere) A solid ball in R3 15 Tangent space of a manifold TM(p) V = vector space p M M … k-manifold The tangent space of the manifold M in p (every point p on the manifold has a different tangent space) is isomorphic to a subspace of V. 16 Tangent space of a manifold V = vector space with dimension n TM(p) p M M … k-manifold . TM(p) is a vector space (subspace of V) and has dimension k. 1-manifold (curves) -> 1 dim TM (lines) . 2-manifold (surface) -> 2 dim TM (planes) . 3-manifold (volumes, e.g. 3-sphere) -> 3 dim TM (full volumes) 17 Tangent space of SO(2) and SO(3) ℝ2×2 ℝ3×3 SO(2) SO(3) 1-manifold 3-manifold TSO(2) is a vector space TSO(3) is a vector space with dimension 1 with dimension 3 3×3 subspace of ℝ2×2 subspace of ℝ subspaces are defined by matrices 18 Skew-symmetric matric . M is skew-symmetric iff MT=-M 0 3 6 0 4 푇 푀 = −푀 −3 0 −1 −4 0 −6 1 0 푠표(푛) = 푀 ∈ ℝ푛×푛|푀푇 = −푀 , +, [] Special orthogonal Lie algebra . The term “algebra” means that so(n) is a vector space (more specific than group) . We have addition in the vector space now (in SO(n) it was only multiplication) 19 Skew-symmetric matrices 푇푆푂 푛 퐼 = 푠표(푛) I SO(n) SO(n) so(n) so(n) R 푇푆푂 푛 푅 = 푅 × 푠표(푛) . The special orthogonal Lie algebra is the tangent space of SO(n) at identity. The tangent space of SO(n) in any other point R is a rotated version of so(n) . TSO(n)(R) is not a skew-symmetric matrix anymore, but a rotated one 20 so(2) and so(3) . so(3) is a vector space of dimension 3 0 3 6 0 4 −3 0 −1 −4 0 . so(2) is a vector space of dimension 1 −6 1 0 I SO(2) I SO(3) Tso(2) Tso(3) is a volume 21 The hat operator . The hat operator is used to form skew-symmetric matrices . for so(3): 0 −푧 푦 . : ℝ3 푠표(3) (푥 , 푦, 푧) 푧 0 −푥 −푦 푥 0 . for so(2): 0 −푥 . : ℝ 푠표(2) 푥 푥 0 . Different notation 0 −푡푧 푡푦 . 푡 푥 = 푡푧 0 −푡푥 −푡푦 푡푥 0 22 The hat operator . The hat operator is used to define the cross-product in matrix form 푎 × 푏 = 푎 푏 ⋁푎, 푏 ∈ ℝ3 23 Lie groups . GL(n), O(n), SO(n) and SE(n) are all Lie groups (groups which are a smooth manifold where the operation is a differentiable function between manifolds) . Also we have seen that the group of skew-symmetric matrices is called Lie algebra so(n) and is the tangent space of the special orthonormal group SO(n) . But how to compute an element of the tangent space so(n) from SO(n) or vice versa? . The exponential map! 24 Exponential map . Given a Lie group G, with its related Lie algebra g=TG(I), there always exists a smooth map from Lie algebra g to the Lie group G called exponential map 푒푥푝: 푔 퐺 I 퐴 SO(n) so(n) exp(퐴) 25 Exponential map . I ℝ푘 휔 ∈ ℝ푘 휔 ∈ 푠표(푛) SO(n) so(n) ∞ 1 exp(휔 ) = 휔 푘 ∈ 푆푂(푛) 푘! exp 퐴 : 푠표(푛) 푆푂(푛) 푘=0 휔 ∈ ℝ푘 exp(휔 ) . Angle-axis representation for rotations: 휔… is the angle-axis representation (ℝ3) exp(휔) is the 3x3 rotation matrix (element of SO(3)) 26 Euler angles . Euler’s Theorem for rotations: Any element in SO(3) can be described as a sequence of three rotations around the canonical axes, where no successive rotations are about the same axis. 1 0 0 푅푥 훼 = 0 cos 훼 −sin 훼 0 sin 훼 cos 훼 cos 훽 0 sin 훽 푅푦 훽 = 0 1 0 −sin 훽 0 cos 훽 cos 훾 −sin 훾 0 푅푧 훾 = sin 훾 cos 훾 0 0 0 1 퐹표푟 푎푛푦 푅 ∈ 푆푂 3 푡ℎ푒푟푒 ∃훼, 훽, 훾 | 푅 = 푅푥(훼)푅푦(훽) 푅푧(훾) . are called Euler angles of R according to the XYZ representation (3 DOF/parameters) 27 Euler angles . Given M (element of SO(3)) there are 12 possible ways to represent it 푀 ∈ 푆푂 3 푡ℎ푒푟푒 ∃훼, 훽, 훾 | 푀 = 푅푥(훼)푅푦(훽) 푅푧(훾) 푀 ∈ 푆푂 3 푡ℎ푒푟푒 ∃훼, 훽, 훾 | 푀 = 푅푥(훼)푅푧(훾)푅푦(훽) 푀 ∈ 푆푂 3 푡ℎ푒푟푒 ∃훼, 훽, 훾 | 푀 = 푅푥(훼)푅푧(훾)푅푥(훽) 푀 ∈ 푆푂 3 푡ℎ푒푟푒 ∃훼, 훽, 훾 | 푀 = 푅푧(훼)푅푥(훾)푅푧(훽) … . A common convention is ZYX corresponding to a rotation first around the x-axis (roll), then the y-axis (pitch) and finally around the z-axis (yaw) 28 [https://en.wikipedia.org/wiki/Aircraft_principal_axes, CC BY-SA 3.0] Euler angles . The parameterization has singularities, called gimbal lock . A gimbal lock happens when after a rotation around an axis, two axes align, resulting in a loss of one degree of freedom cannot move (gimbal lock) 29 Angle-Axis . Euler’s rotation theorem also states that any rotation can be expressed as a single rotation about some axis.