Noname manuscript No. (will be inserted by the editor) Rotation Averaging Richard Hartley, Jochen Trumpf, Yuchao Dai, Hongdong Li Received: date / Accepted: date Abstract This paper is conceived as a tutorial on rota- Single rotation averaging. In the single rotation averaging tion averaging, summarizing the research that has been car- problem, several estimates are obtained of a single rotation, ried out in this area; it discusses methods for single-view which are then averaged to give the best estimate. This may and multiple-view rotation averaging, as well as providing be thought of as finding a mean of several points Ri in the proofs of convergence and convexity in many cases. How- rotation space SO(3) (the group of all 3-dimensional rota- ever, at the same time it contains many new results, which tions) and is an instance of finding a mean in a manifold. were developed to fill gaps in knowledge, answering funda- Given an exponent p ≥ 1 and a set of n ≥ 1 rotations p mental questions such as radius of convergence of the al- fR1;:::; Rng ⊂ SO(3) we wish to find the L -mean rota- gorithms, and existence of local minima. These matters, or tion with respect to d which is defined as even proofs of correctness have in many cases not been con- n p X p sidered in the Computer Vision literature. d -mean(fR1;:::; Rng) = argmin d(Ri; R) : We consider three main problems: single rotation av- R2SO(3) i=1 eraging, in which a single rotation is computed starting Since SO(3) is compact, a minimum will exist as long as the from several measurements; multiple-rotation averaging, in distance function is continuous (which any sensible distance which absolute orientations are computed from several rel- function is). This problem has been much studied in the lit- ative orientation measurements; and conjugate rotation av- erature, but there are still open problems, some of which are eraging, which relates a pair of coordinate frames. This resolved here. last is related to the hand-eye coordination problem and to multiple-camera calibration. Conjugate rotation averaging. In the conjugate rotation averaging problem, n ≥ 1 rotation pairs (Li; Ri) (the left Keywords geodesic distance · angular distance · chordal and right rotations) are given, and we need to find a rota- −1 distance · quaternion distance · L1 mean · L2 mean · tion S such that Ri = S LiS for all i. This problem arises conjugate rotation when the rotations Ri and Li are measured in different coor- dinate frames, and the coordinate transformation S that re- lates these two frames is to be determined. In the presence of noise, the appropriate minimization 1 Introduction problem is then to find n X −1 p In this paper, we will be interested in three different rotation argmin d(Ri; S LiS) : S averaging problems. In the following description, d(R; S) de- i=1 notes the distance between two rotations R and S. Various This problem is sometimes referred to as the hand-eye coor- different possible distance functions will be described later dination problem, see e.g. [13,64,84]. in the paper; for now, d(·; ·) is thought of as being any arbi- In the case where the individual rotations Ri and Li trary metric on the space of rotations SO(3). are themselves estimated from relative orientation measure- ments Rij and Lij, the two problems can be solved simulta- Address(es) of author(s) should be given neously to find S at the same time as the rotations (Ri; Li). 2 Multiple rotation averaging. In the multiple rotation av- In another application, camera rigs used in robotic or eraging problem, several relative rotations Rij are given, per- mapping applications can consist of fixed cameras often haps relating different coordinate frames, and n absolute with small or no overlap of fields of view. From SfM tech- rotations Ri are computed to satisfy the compatibility con- niques, the trajectory of each camera may be computed in- straint RijRi = Rj. Only some Rij are given, represented by dependently. In the two-camera case this leads to pairs of index pairs (i; j) in a set N . In the presence of noise, the rotations (Li; Ri). By solving the conjugate averaging prob- appropriate minimization problem is expressed as seeking lem, one may compute the relative orientation of the two cameras. This technique generalizes easily to several cam- X −1 p argmin d(Rij; RjRi ) : eras. For best results, the conjugate averaging problem is R ;:::;R 1 n (i;j)2N solved simultaneously with the multiple-rotation averaging For all these problems, we are interested in finding prov- problem of determining the Ri and Li [12]. ably optimal and convergent solutions, mainly for the cases p = 1 and p = 2. This includes most particularly identi- Different metrics. Although the rotation averaging prob- fying the conditions under which the problems will allow a lem has been discussed frequently in the literature of Com- solution. puter Vision, there has rarely been any discussion of what Our task in this paper is to report the known results about cost-function is actually being minimized by the algorithms these problems, while at the same time filling in gaps of in question. Discussion of this question in papers about opti- knowledge, particularly related to convergence, convexity or mization on manifolds has usually been more specific in this uniqueness of solutions to these problems. regard. The most common approach to the single-averaging problem is to find the Karcher mean [26,40] which is de- fined as Applications. The single-rotation averaging problem can be used in the case where several measurements of a single n ∗ X 2 y = argmin dgeod(xi; y) (1) rotation R are given. These may be for instance measure- y ments of the orientation of an object, derived from measure- i=1 ments taken with different cameras in a calibrated network. where xi; i = 1; : : : ; n are several points on a Riemannian If the measurements are noisy, they can be averaged to find manifold, and dgeod(·; ·) represents the minimal geodesic a mean. In another example, given a pair of images, several distance between two points. The choice of the squared- minimal sets of points (5 points for calibrated cameras) may distance in this expression means that we are minimizing be chosen and used to compute the relative rotation between a least-squares (L2) cost function. This definition is eas- the cameras. By a process of averaging, one may obtain the ily generalized to include other than sum of squares costs. mean of these measurements, which provides an estimate of The most immediate generalization is to minimize the L1 the true rotation relating the two cameras. cost, namely the sum of (unsquared) distances dgeod(xi; y). The multiple-rotation averaging problem has wide appli- We will refer to this as the geodesic L1-mean of the points. q cation to the problem of structure-from-motion (SfM), and Other exponents, such as dgeod(xi; y) are possible, but will several papers [55,73,29,37,71,41,38] have explored this not be considered in any detail in this paper. Thus, by refer- method, often starting with an assumption that the rotations ring to a geodesic mean, we imply the minimization of a cost of the cameras are known. These rotations may be estimated based on geodesic distance in the manifold itself. The liter- separately by rotation averaging. This idea has been devel- ature on the Karcher mean is very large, see e.g. [26,40,11, oped into a unified approach to SfM by Govindu [23,22,24], 47,2] and the references therein. Papers relating to compu- who also developed various rotation-averaging algorithms. tation of the Karcher mean for rotations include [57], [48], Conjugate rotation averaging is related to the hand-eye [53] and [45], with Manton [53] giving a simple iterative coordination problem, common in robotics [13,64,84]. In solution. one formulation of this problem, consider a robot manipulat- Computation of the geodesic L1-mean in a manifold has ing some object, which is also observed by a stationary cam- received much less attention. Recent work includes L1 min- era. The orientation of the object can be computed at each imization on SO(3) [12], which suggests a gradient-descent moment through knowledge of the geometry of the robot algorithm. This problem has been solved in the more general (for instance, joint-angles). At the same time, the orienta- context of a Riemannian manifold with positive sectional tion of the object can be computed from the images taken curvature in [20] and extended in [83]. The solution of [20] from the camera. This gives two separate estimates of the involves iterative steps of the Weiszfeld algorithm [81] in orientation of the object (expressed as a rotation), but these tangent spaces of the manifold. This literature will be sur- are in different coordinate frames. By solving the conjugate veyed in more detail later. rotation problem, one can compute the relationship between In the context of rotations in SO(3), the (natural) the robot and camera frames. geodesic metric dgeod(·; ·) is equal to the angle between two 3 rotations. Specifically, given rotations R and S, the product simply to be locally shortest paths on a manifold; all the re- RS−1 is also a rotation, about some axis by an angle θ in the quired properties are derived using elementary concepts. range 0 ≤ θ ≤ π. We define d\(R; S) = θ, and refer to it as Since the word “manifold” itself is often used in Com- the angle metric.