between subspaces

Eckhard Hitzer, Department of Applied , University of Fukui, 910-8507 Japan June 10, 2013

Abstract We first review the definition of the between subspaces and how it is computed using . Then we introduce the Grassmann and Clifford algebra description of subspaces. The geometric product of two subspaces yields the full relative angular information in an explicit manner. We explain and interpret the result of the geometric product of subspaces gaining thus full access to the relative information.

Keywords: Clifford , subspaces, relative angle, principal angles, principal vectors. AMS Subj. Class.: 15A66.

1 Introduction lines, planes, circles ans spheres of three dimen- sional [7] and was able to find I first came across Clifford’s geometric algebra in the one general formula fully expressing the relative early 90ies in papers on gauge field theory of gravity orientation of any two of these objects. Yet L. by J.S.R. Chisholm, struck by the seamlessly com- Dorst (Amsterdam) later asked me if this formula pact, elegant, and geometrically well interpretable could be generalized to any , because expressions for elementary particle fields subject to by his experience formulas that work dimension Einstein’s gravity. Later I became familiar with D. independent are right. I had no immediate answer, Hestenes’ excellent modern formulation of geometric it seemed to complicated to me, having to deal with algebra, which explicitly shows how the geometric too many possible cases. product of two vectors encodes their complete rela- But when I prepared for December 2009 a presen- tive orientation in the inner product part (co- tation on neural computation and Clifford algebra, I sine) and in the part (sine). came across a 1983 paper by Per Ake Wedin on an- Geometric algebra can be viewed as an algebra of gles between subspaces of finite dimensional inner a vector and all its subspaces, represented by product spaces [2], which taught me the classical socalled blades. I therefore often wondered if the approach. In addition it had a very interesting note geometric product of subspace blades also encodes on solving the problem, essentially using Grassmann their complete relative orientation, and how this is algebra with an additional canonically defined inner done? What is the form of the result, how can it product. After that the various bits and pieces came together and began to show the whole picture, the arXiv:1306.1629v1 [math.MG] 7 Jun 2013 be interpreted and put to further use? I learned more about this problem, when I worked on the picture which I want to explain in this contribution. conformal representation of points, point pairs, Permission to make digital or hard copies of all or part of this work for personal or classroom use is 2 The angle between two lines granted without fee provided that copies are not made or distributed for profit or commercial ad- vantage and that copies bear this notice and the To begin with let us look (see Fig. 1) at two lines A, B in a Rn, which are spanned by two full citation on the first page. To copy otherwise, n or republish, to post on servers or to redistribute (unit) vectors a, b ∈ R , a · a = b · b = 1: to lists, requires prior specific permission and/or a fee. A = span{a}, B = span{b}. (1) Example: 2 Lines

tween the subspaces A, B is characterized by a set of r principal angles θk, 1 ≤ k ≤ r, as indicated in b Fig. 2. A principal angle is the angle between two principal vectors a k ∈ A and bk ∈ B. The spanning sets of vectors {a 1,..., a r}, and {b1,..., br} can A,B a be chosen such that pairs of vectors a k, bk either • agree a k = bk, θk = 0,

• or enclose a finite angle 0 < θk ≤ π/2. Figure 1:• Two Angle linesθA,B betweengiven b twoy two lines unitA, B vectors, spanned by unit vectors a, b, respectively. Angles– Line between A: spanned Subs by vectorpaces a A,BIn addition the pairs of vectors {a k, bk}, 1 ≤ k ≤ r span mutually orthogonal lines (for θk = 0) and – Line B: spanned by vector b (principal) planes ik (for 0 < θk ≤ π/2). These mu- tually orthogonal planes ik are indicated in Fig. 2. • Angle 0 ≤ A,B ≤ between lines A,B b1 br Therefore if a k ∦ bk and a l ∦ bl for 1 ≤ k 6= l ≤ r, b2 b3cos A,B = a·b then ik is orthogonal to plane il. The cosines  B of the socalled principal angles θk may therefore 1 r be cos θk = 1 (for a k = bk), or cos θk = 0 (for 2 3 a k ⊥ bk), or any value 0 < cos θk < 1. The total a1 ar angle between the two subspaces A, B is defined as A a the product i1 a2 3 ir i3 cos θA,B = cos θ1 cos θ2 ... cos θr. (4) i2 In this definition cos θA,B will automatically be zero A = span(a ,a ,a ,…,a ), B=span(b ,b , b , …if, any b ) pair of principal vectors {a k, bk}, 1 ≤ k ≤ r is Figure 2: Angular1 2 relationship3 r of two subspaces1 2A, 3 .r Then the two subspaces are said to B, spanned by two sets of vectors {a ,..., a }, and cos A,B = cos 1 cos 2 cos1 3 … cosr r be perpendicular A ⊥ B, a familiar notion from three {b1,..., br}, respectively. , where two perpendicular planes A, B share a common line spanned by a 1 = b1, and have two mutually orthogonal principal vectors a 2 ⊥ b2,

The angle 0 ≤ θA,B ≤ π/2 between lines A and B is which are both in turn orthogonal to the common simply given by line vector a 1. It is further possible to choose the indexes of the vector pairs {a k, bk}, 1 ≤ k ≤ r such cos θA,B = a · b. (2) that the principle angles θk appear ordered by mag- nitude θ1 ≥ θ2 ≥ ... ≥ θr. (5) 3 Angles between two sub- spaces (described by princi- 4 Matrix algebra computation pal vectors) of angle between subspaces

Next let us examine the case of two r-dimensional The conventional method of computing the angle (r ≤ n) subspaces A, B of an n-dimensional Eu- θA,B between two r-dimensional subspaces A, B ⊂ n n 0 0 0 clidean vector space R . The situation is depicted R spanned by two sets of vectors {a 1, a 2,... a r} 0 0 0 in Fig. 2. Each subspace A, B is spanned by a set and {b1, b2,... br} is to first arrange these vectors of r linearly independent vectors as column vectors into two n × r matrices

n 0 0 0 0 A = span{a 1,..., a r} ⊂ R , MA = [a 1,..., a r],MB = [b1,..., br]. (6) n B = span{b1,..., br} ⊂ R . (3) Then standard matrix algebra methods of QR de- Using Fig. 2 we introduce the following notation composition and singular value decomposition are for principal vectors. The angular relationship be- applied to obtain • r pairs of singular unit vectorsVectorsa k, bk and and b a • r singular values σ = cos θ = a · b . oriented k k k k unit  = This approach is very computation1+a2 e2, b intensive.b1e1+b2e2 a b (a1b2  a2b1) e1e2 = |a||b| sin e1e2 5 Even more subtle b∧a …oriented ways area spanned by a,b 0 ⇔ a || b ⇔ b =  a , ∊R Per Ake Wedin in his 1983th direction contribution a : {x [2] ∊R ton | ax∧a= 0} conference on Matrix Pencils entitled On Angles between Subspaces of ar Finites can be Dimensional freely reshaped first carefully treatsa∧ theb = abovea∧(b men-+ a) a tioned matrix algebra approacha product to of computing orthogonal the vectors angle θA,B in great detail and clarity. Towards the b’ end of his paper he dedicatesa∧b less= a thanb’ , onea  pageb’ (a to · b’ = 0 ) mentioning an alternative method starting out with a the words: But there are even more subtle ways to define angle functions. Figure 3: Bivectors a ∧ b as oriented area elements There he essentially reviews how r-dimensional can be reshaped (e.g. by b → b + µa, µ ∈ R) with- subspaces A, B ⊂ Rn can be represented by r-vectors out changing their value (area and orientation). The (blades) in Grassmann algebra A, B ∈ Λ(Rn): bottom figure shows orthogonal reshaping into the form of an oriented rectangle. n A = {x ∈ R |x ∧ A = 0}, n B = {x ∈ R |x ∧ B = 0}. (7)

n 6 Clifford (geometric) algebra The angle θA,B between the two subspaces A, B ∈ R can then be computed in a single step Clifford (geometric) algebra is based on the geomet- ric product of vectors a, b ∈ p,q, p + q = n A · Be R cos θA,B = = cos θ1 cos θ2 ... cos θr, (8) |A||B| ab = a · b + a ∧ b, (9) where the inner product is canonically defined on and the associative algebra Cl thus generated the Grassmann algebra Λ( n) corresponding to the p,q R with and p,q as subspaces of Cl . a · b is geometry of n. The tilde is the reverse R R p,q R the symmetric inner product of vectors and a ∧ b operation representing a dimension dependent r(r−1) is Grassmann’s outer product of vectors represent- change Be = (−1) 2 B, and |A| represents the ing the oriented parallelogram area spanned by a, b, 2 of A, i.e. |A| = A · Ae, and similarly compare Fig. 3. |B|2 = B · Be. Wedin refers to earlier works of L. As an example we take the Clifford geometric al- Andersson [1] in 1980, and a 1963 paper of Q.K. Lu gebra Cl3 = Cl3,0 of three-dimensional (3D) Eu- [5]. clidean space R3 = R3,0. R3 has an orthonormal n Yet equipping a Grassmann algebra Λ(R ) with a {e1, e2, e3}. Cl3 then has an eight-dimensional canonical inner product comes close to introducing basis of n Clifford’s geometric algebra Cln = Cl(R ). And there is another good reason to do that, as e.g. H. {1, e1, e2, e3, e2e3, e3e1, e1e2, i = e1e2e3 }. (10) Li explains in his excellent 2008 textbook Invariant | {z } | {z } | {z } vectors area bivectors volume trivector and Geometric Reasoning [4]: ... to allow sums of angles to be advanced invariants, the inner- Here i denotes the unit trivector, i.e. the oriented product Grassmann algebra must be extended to the volume of a unit cube, with i2 = −1. The even + Clifford algebra ... This is why I have decided to grade subalgebra Cl3 is isomorphic to Hamilton’s + immediately begin in the next section with Clifford’s H. Therefore elements of Cl3 are also geometric algebra instead of first reviewing inner- called rotors ( operators), rotating vectors product Grassmann algebra. and of Cl3. In general Clp,q, p+q = n is composed of so-called Whereas the antisymmetric outer product part gives r-vector subspaces spanned by the induced bases the bivector, which represents the oriented area of the parallelogram spanned by a and b

{ek1 ek2 ... ekr | 1 ≤ k1 < k2 < . . . < kr ≤ n}, (11) a ∧ b = (a1b2 − a2b1)e1e2 = |a||b| sin θa,b e1e2. r (18) each with dimension n . The total dimension of Pn r n The parallelogram has the (signed) scalar area the Clp,q therefore becomes = 2 . r=0 n n General elements called multivectors M ∈ |a||b| sin θa,b and its orientation in the space R is given by the oriented unit area bivector e1e2. Clp,q, p + q = n, have k-vector parts (0 ≤ k ≤ n): Two non-zero vectors a and b are parallel, if and scalar part Sc(M) = hMi = hMi0 = M0 ∈ R, vec- p,q only if a ∧ b = 0, i.e. if and only if sin θ = 0 tor part hMi1 ∈ R , bi-vector part hMi2, . . . , and a,b Vn p,q part hMin ∈ R a ∧ b = 0 ⇔ a k b ⇔ b = αa, α ∈ R. (19) 2n X We can therefore use the outer product to represent M = MAeA = hMi+hMi1 +hMi2 +...+hMin . a line A = span{a} with direction vector a ∈ Rn as A=1 (12) n A = {x ∈ R | x ∧ a = 0}. (20) The reverse of M ∈ Clp,q defined as Moreover, bivectors can be freely reshaped (see Fig. n X k(k−1) 3), e.g. Mf = (−1) 2 hMik, (13) k=0 a ∧ b = a ∧ (b + µa), µ ∈ R, (21) often replaces complex conjugation and because due to the antisymmetry a ∧a = 0. This re- conjugation. Taking the reverse is equivalent to re- shaping allows to (orthogonally) reshape a bivector versing the order of products ob basis vectors in the to rectangular shape basis blades of (11). For example the reverse of the 0 0 bivector e1e2 is a ∧ b = ab , a ⊥ b(i.e. a · b = 0) (22)

e]1e2 = e2e1 = −e1e2, (14) as indicated in Fig. 3. The shape may even chosen as or circular, depending on the application because only the antisymmetric outer product part in mind. e2e1 = e2 ∧ e1 = −e1 ∧ e2 is relevant. The total antisymmetry of the trivector x ∧ a ∧ b The scalar product of two multivectors M, Ne ∈ means that Clp,q is defined as x ∧ a ∧ b = 0 ⇔ x = αa + βb, α, β ∈ R. (23) M ∗ N = hMNi = hMNi . (15) e e e 0 Therefore a plane B is given by a simple bivector (also called 2-blade) B = a ∧ b as For M, Ne ∈ Cln = Cln,0 we get M ∗ Ne = P M N . The modulus |M| of a M ∈ n A A A B = {x ∈ R | x ∧ B = 0}. (24) Cln is defined as A three-dimensional volume subspace C is simi- 2 X 2 |M| = M ∗ Mf = MA. (16) larly given by a 3-blade C = a ∧ b ∧ c as A n C = {x ∈ R | x ∧ C = 0}. (25)

6.1 Subspaces described in geometric Finally a blade Dr = b1 ∧ b2 ∧ ... ∧ br, bl ∈ algebra Rn, 1 ≤ l ≤ r ≤ n describes an r-dimensional vector subspace In Cln symmetric inner product part of two vectors p,q a = a1e1+a2e2, b = b1e1+b2e2 yields the expected D = {x ∈ R |x ∧ D = 0}. (26) result Its dual blade ∗ −1 a · b = a1b1 + a2b2 = |a||b| cos θa,b . (17) D = Din (27) describes the complimentary (n − r)-dimensional therefore ⊥ vector subspace D . The of the blade |Ar| = |a 1| ... |a r|. (36) D ∈ Cl is nothing but the volume of the r- r n Every r-blade A can therefore be written as a dimensional parallelepiped spanned by the vectors r product of the scalar magnitude |A | times the ge- {b , b ,..., b }. r 1 2 r ometric product of exactly r mutually orthogonal Just as we were able to orthogonally reshaped a unit vectors {a ,..., a } bivector to rectangular or square shape we can re- b1 br shape every r-blade A to a geometric product of r Ar = |Ar|ab1ab2 ... abr. (37) mutually orthogonal vectors Please note well, that this rewriting of an r-blade 0 0 0 Ar = a 1 ∧ a 2 ∧ ... a r = a 1a 2 ... a r, (28) in geometric algebra does not influence the overall with pairwise orthogonal and anticommuting vec- result on the left side, the r-blade Ar is before and after the rewriting the very same element of the geo- tors a 1 ⊥ a 2 ⊥ ... ⊥ a r. The reverse Aer of the geometric product of orthogonal vectors A = algebra Cln. But for the geometric interpre- r tation of the geometric product AB of two r-blades a 1a 2 ... a r is therefore clearly A, B ∈ Cln the orthogonal reshaping is indeed a key ∼ r(r−1) step. (a 1a 2 ... a r) = a r ... a 2a 1 = (−1) 2 a 1a 2 ... a r, (29) After a short discussion of reflections and rota- r(r−1) tions implemented in geometric algebra, we return by simply counting the number of permuta- 2 to the geometric product of two r-blades A, B ∈ Cl tions necessary. n and present our key insight. Paying attention to the dimensions we find that the outer product of an r-blade B with a vector a increases the dimension (grade) by +1 6.2 Reflections and rotations A simple application of the geometric product is a ∧ B = haBir+1. (30) shown in Fig. 4 (left) to the reflection of a point Opposite to that, the inner product (or left contrac- vector x at a plane with vector a, which tion) with a vector lowers the dimension (grade) by means to reverse the component of x parallel to a −1 (perpendicular to the plane) a · B = haBi . (31) r−1 a x −→ x 0 = −a −1xa, a −1 = . (38) The geometric product of two r-blades A, B contains a 2 therefore at most the following grades Two reflections lead to a rotation by twice the angle

AB = hABi0 +hABi2 +...+hABi2min(r,[n/2]), (32) between the reflection planes as shown in Fig. 4 (right) where the limit [n/2] (entire part of n/2) is due to 00 −1 −1 −1 the dimension limit of Rn. x −→ x = a b xab = (ab)xab = R xR, The inner product of vectors is properly general- (39) ized in geometric algebra by introducing the (left) with rotation operator () R = ab ∝ cos θa,b + contraction of the r-blade A = Ar onto the s-blade ia,b sin θa,b , where the unit bivector ia,b represents B = Bs as the . ArcBs = hABis−r. (33) For blades of equal grade (r = s) we thus get the 7 Geometric information in symmetric scalar the geometric product of two ArcBr = hABi0 = hBAi0 = A ∗ B. (34) subspace r-blades Finally the product of a blade with its own re- verse is necessarily scalar. Introducing orthogonal From the foregoing discussion of the representation n reshaping this scalar is seen to be of r-dimensional subspaces A, B ⊂ R by the blades 0 0 0 0 A = a 1 ∧ ... a r and B = b1 ∧ ... br, from the free- 2 2 2 ArAer = a 1 ... a ra r ... a 1 = a 1 ... a r = |Ar| , dom of orthogonally reshaping these blades and fac- (35) toring out the blade magnitudes |A| and |B|, and − x' |A||B|(s1c2 . . . cri1 +c1s2 . . . cri2 +...+c1c2 . . . srir), x|| x|| α etc., up to the 2r-vector (or 2[n/2]-vector) part x' x x'' x x⊥ hABei2r = |A||B|s1s2 . . . sri1i2 ... ir of the geometric product ABe arise and what information they carry. a b α/2 Obviously the scalar part yields the cosine of the angle between the subspaces represented by the two a r-vectors A, B ∈ Cln

hABei0 A ∗ Be cos θ = = , (43) AB |A||B| |A||B| Figure 4: Reflection at a plane with normal a (left) and rotation as double reflection at planes normal which exactly corresponds to P.A. Wedin’s formula to a, b (right). from inner-product Grassmann algebra. The bivector part consists of a sum of (principal) plane bivectors, which can in general be uniquely from the classical results of matrix algebra, we now decomposed into its constituent sum of 2-blades by know that we can rewrite the geometric product AB the method of Riesz, described also in [6], chapter in mutually orthogonal products of pairs of principal 3-4, equation (4.11a) and following. vectors a k, bk, 1 ≤ k ≤ r The magnitude of the 2r-vector part allows to compute the product of all sines of principal angles ABe = a 1a 2 ... a rbr ... b2b1 = a 1b1a 2b2 ... a rbr. (40) |hABei2r| s1s2 . . . sr = ± . (44) The geometric product |A||B|

a b = |a ||b |(cos θ + i sin θ ) r r r r a r ,b r a r ,b r a r ,b r Let us finally refine our considerations to two gen- eral r-dimensional subspaces A, B, which we take to = |a r||br|(cr + irsr), (41) partly intersect and to be partly perpendicular. We with c = cos θ and s = sin θ in the r a r ,b r r a r ,b r mean by that, that the dimension of the intersect- above expression for ABe is composed of a scalar ing subspace be s ≤ r (s is therefore the number and a bivector part. The latter is proportional to of principal angles equal zero), and the number of the unit bivector ir representing a (principal) plane principle angles with value π/2 be t ≤ r − s. For orthogonal to all other principal vectors. ir there- simplicity we work with normed blades (i.e. after fore commutes with all other principal vectors, and dividing with |A||B|. The geometric product of the hence the whole product a rbr (a rotor) commutes the r-blades A, B ∈ Cln then takes the form with all other principal vectors. A completely analo- gous consideration applies to all products of pairs of ABe principal vectors, which proofs the second equality = (cs+1cs+2 . . . cr−t+ in (40). + ss+1cs+2 . . . cr−tis+1 + cs+1ss+2 . . . cr−tis+2+ We thus find that we can always rewrite the prod- ... + c c . . . s i + uct ABe as a product of rotors s+1 s+2 r−t r−t . ABe . + s s . . . s i i ... i )i ... i . = |A||B|(c1 + i1s1)(c2 + i2s2) ... (cr + irsr) s+1 s+2 r−t s+1 s+2 r−t r−t+1 r (45) = |A||B|(c1c2 . . . cr+ + s1c2 . . . cri1 + c1s2 . . . cri2 + ... + c1c2 . . . srir+ We thus see, that apart from the integer dimensions . s for parallelity (identical to the dimension of the . meet of blade A with blade B) and t for perpendic- + s1s2 . . . sri1i2 ... ir) (42) ularity, the lowest non-zero grade of dimension 2t gives the relevant angular measure We realize how the scalar part hABei0 = |A||B|c1c2 . . . cr, the bivector part hABei2 = cos θAB = cos θs+1 cos θs+2 ... cos θr−t. (46) While the maximum grade part gives again the [3] L. Dorst et al, Geometric Algebra for Comp. product of the corresponding sinuses Sc., Morgan Kaufmann, 2007.

sin θs+1 sin θs+2 ... sin θr−t. (47) [4] H. Li, Invariant Algebras and Geometric Rea- soning, World Scientific, Singapore, 2008. Dividing the product ABe by its lowest grade part cs+1cs+2 . . . cr−tir−t+1 ... ir gives a multivector with [5] Q.K. Lu, The elliptic geometry of extended maximum grade 2(r − t − s), scalar part one, and spaces, Acta Math. Sinica, 13 (1963), pp. 49– bivector part 62; translated as Chinese Math. 4 (1963), pp. 54–69. ts+1is+1 + ts+2is+2 + ... + tr−tir−t, (48) [6] D. Hestenes, G. Sobczyk, Clifford Algebra to where tk = tan θk. Splitting this bivector into its Geometric Calculus, Kluwer, 1984. constituent bivector parts further yields the (prin- cipal) plane bivectors and the tangens values of the [7] E. Hitzer, K. Tachibana, S. Buchholz, I. Yu Carrier method for the general evaluation and principle angles θk, s < k ≤ r − t. This is the only somewhat time intensive step. control of pose, molecular conformation, track- ing, and the like Advances in Applied Clifford Algebras, Vol. 19(2), pp. 339-364 (2009). 8 Conclusion

Let us conclude by discussing possible future appli- cations of these results. The complete relative orien- tation information in ABe should be ideal for a sub- space structure self organizing map (SOM) type of neural network. Not only data points, but the topol- ogy of whole data subspace structures can then be faithfully mapped to lower dimensions. Our discus- sion gives meaningful results for partly intersecting and partly perpendicular subspaces. Apart from ex- tracting the bivector components, all computations are done by multiplication. Projects like fast Clif- ford algebra hardware developed at the TU Darm- stadt (D. Hildenbrand et al) should be of interest for applying the results of the paper to high dimen- sional data sets. An extension to offset subspaces (of ) and r-spheres (of confor- mal geometric algebra) may be possible.

Acknowledgments

Soli deo gloria. I do thank my dear family, H. Ishi, D. Hildenbrand and V. Skala.

References

[1] L. Andersson, The concepts of angle between subspaces ... unpublished notes, Umea (1980). [2] P. A. Wedin, On angles between subspaces of a finite-dimensional inner product space, in Ma- trix Pencils, Bo Kagstram and Axel Ruhe, eds., Springer-Verlag, Berlin, 1983, pp. 263–285.