1 Geometric-Algebra Adaptive Filters Wilder B. Lopes∗, Member, IEEE, Cassio G. Lopesy, Senior Member, IEEE Abstract—This paper presents a new class of adaptive filters, namely Geometric-Algebra Adaptive Filters (GAAFs). They are Faces generated by formulating the underlying minimization problem (a deterministic cost function) from the perspective of Geometric Algebra (GA), a comprehensive mathematical language well- Edges suited for the description of geometric transformations. Also, (directed lines) differently from standard adaptive-filtering theory, Geometric Calculus (the extension of GA to differential calculus) allows Fig. 1. A polyhedron (3-dimensional polytope) can be completely described for applying the same derivation techniques regardless of the by the geometric multiplication of its edges (oriented lines, vectors), which type (subalgebra) of the data, i.e., real, complex numbers, generate the faces and hypersurfaces (in the case of a general n-dimensional quaternions, etc. Relying on those characteristics (among others), polytope). a deterministic quadratic cost function is posed, from which the GAAFs are devised, providing a generalization of regular adaptive filters to subalgebras of GA. From the obtained update rule, it is shown how to recover the following least-mean squares perform calculus with hypercomplex quantities, i.e., elements (LMS) adaptive filter variants: real-entries LMS, complex LMS, that generalize complex numbers for higher dimensions [2]– and quaternions LMS. Mean-square analysis and simulations in [10]. a system identification scenario are provided, showing very good agreement for different levels of measurement noise. GA-based AFs were first introduced in [11], [12], where they were successfully employed to estimate the geometric Index Terms—Adaptive filtering, geometric algebra, quater- transformation (rotation and translation) that aligns a pair of nions. 3D point clouds (a recurrent problem in computer vision), while keeping a low computational footprint. This work takes a I. INTRODUCTION step further: it uses GA and GC to generate a new class of AFs DAPTIVE filters (AFs), usually described via linear able to naturally estimate hypersurfaces, hypervolumes, and A algebra (LA) and standard vector calculus [1], can be elements of greater dimensions (multivectors), generalizing interpreted as instances for geometric estimation, since the regular AFs from the literature. Filters like the regular least- weight vector to be estimated represents a directed line in an mean squares (LMS) – real entries, the Complex LMS (CLMS) underlying vector space, i.e, a n-dimensional vector encodes – complex entries [13], and the Quaternion LMS (QLMS) – the length and direction of an edge in a n-dimensional polytope quaternion entries [14]–[16], are recovered as special cases of (see Fig. 1). However, to estimate the polytope as a whole the more comprehensive GA-LMS introduced herein. (edges, faces, and so on), a regular AF designed in light of The text is organized as follows. Section II covers the LA might not be very helpful. transition from linear to geometric algebra, presenting the Indeed, LA has limitations regarding the representation of fundamentals of GA and GC. In Section III, the standard geometric structures [2, p. 20]. Take for instance the matrix quadratic cost function, usually adopted in adaptive filtering, product between two vectors: it always results in either a scalar is recovered as a particular case of a more comprehensive or a matrix (2-dimensional array of numbers). Thus, one may quadratic cost function that can only be written using the wonder if it is possible to construct a new kind of product geometric product. In Section IV the gradient of that cost that takes two vectors (directed lines or edges) and returns function is calculated via GC techniques and the GA-LMS arXiv:1608.03450v2 [math.OC] 21 Aug 2018 a hypersurface (the face of an n-dimensional polytope); or is formulated. Section V provides a mean-square analysis takes a vector and a hypersurface and returns another polytope. (steady state) with the support of the energy conservation Similar ideas have been present since the advent of algebra, in relations [1]. Experiments are shown in Section VI to assess an attempt to establish a deep connection with geometry [3], the performance of the GAAFs against the theoretical analysis [4]. in a system-identification scenario. Finally, conclusion remarks The geometric product, which is the product operation of are presented in Section VII. GA [5], [6], captures the aforementioned idea. It allows one Remarks about notation: Table I summarizes the notation to map a set of vectors not only onto scalars, but also onto convention. While those are deterministic quantities, their hypersurfaces and n-dimensional polytopes. The use of GA in- boldface versions represent random quantities. The name array creases the portfolio of geometric shapes and transformations of multivectors was chosen to avoid confusion with vectors in one can represent. Also, its extension to calculus, namely geo- Rn, which in this text have the usual meaning of collection metric calculus (GC), allows for a clear and compact way to of real numbers (row or column). In this sense, an array of multivectors can be interpreted as a “vector” that allows ∗R&D Dept. at UCit (ucit.fr) and OpenGA.org, [email protected]. yDept. of Electronic Systems Engineering, University of Sao Paulo, Brazil, cas- for hypercomplex entries (numbers constructed by adding [email protected]. “imaginary units” to real numbers [17]). 2 TABLE I that two nonparallel directed segments determine a parallel- SUMMARY OF NOTATION. ogram, a notion which can not be described by the inner Symbol Definition product. The multiplication a^b results in an oriented surface a Arrays of multivectors, vectors and scalars. or bivector (see Fig. 2a). Such a surface can be interpreted A General multivector or matrix. as the parallelogram (hyperplane) generated when vector a is r Rotor (type of multivector). Time-varying scalar, vector (or array of multivectors), swept on the direction determined by vector b. Alternatively, a(i); a ;A(i) i and general multivector (or matrix), respectively. the outer product can be defined as a function of the angle θ between a and b a ^ b = Ia;bjajjbjsinθ; (3) II. FROM LINEAR TO GEOMETRIC ALGEBRA 1 where Ia;b is the unit bivector that defines the orientation of To start transitioning from LA to GA, one needs to recall the the hyperplane a ^ b [2, p.66]. Note that in the particular case definition of an algebra over the reals [6], [9], [18]: a vector of 3D Euclidean space, (3) is reduced to the cross product space V over the field R, equipped with a bilinear map V × a × b = pjajjbjsinθ, where p is the unit vector normal to the V ! V denoted by ◦ (the product operation of the algebra), plane containing fa; bg. From Fig. 2a it can be concluded that is said to be an algebra over R if the following relations hold the outer product is noncommutative, i.e., a ^ b = −b ^ a: 8fa; b; cg 2 V and fα; βg 2 R, the orientation of the surface generated by sweeping a along (a + b) ◦ c = a ◦ c + b ◦ c (Left distributivity); b is opposite to the orientation of the surface generated by c ◦ (a + b) = c ◦ a + c ◦ b (Right distributivity); sweeping b along a. 2 (αa) ◦ (βb) = (αβ)(a ◦ b) (Compatibility with scalars). Finally, the geometric product of vectors a and b is denoted (1) by their juxtaposition ab and defined as Linear (matrix) algebra, utilized to describe adaptive fil- ab , a · b + a ^ b; (4) tering theory, is constructed from the definition above. The in terms of the inner (·) and outer (^) products [6, Sec. 2:2]. elements of this algebra are matrices and vectors, which Note that due to a ^ b = −(b ^ a) the geometric product is multiplied among themselves via the matrix product generate noncommutative, resulting in ab = −ba, and it is associative, new matrices and vectors. Additionally, to express the notion a(bc) = (ab)c, fa; b; cg 2 Rn. of vector length and angle between vectors, LA adopts the In linear algebra the fundamental elements are matri- bilinear form V × V ! R, i.e., inner product, which returns ces/vectors. In a similar way, in GA the basic elements are a real number as a result of the multiplication between two the so-called multivectors (Clifford numbers). The structure vectors in V (one says that V is a normed vector space) [18, of a multivector can be seen as a generalization of complex p. 180]. numbers and quaternions for higher dimensions. For instance, Geometric (Clifford) Algebra derives from (1), however a complex number α+jβ has a scalar part α combined with an with a different product operation. Such product, called geo- imaginary part jβ; quaternions like α+iβ1 +jβ2 +kβ3 expand metric product, is what allows for GA to be a mathematical that by adding two extra imaginary-valued parts. Multivectors language that unifies different algebraic systems trying to generalize this structure, which contains one scalar part and express geometric relations/transformations, e.g., rotation and several other parts named grades. Thus, a general multivector translation. The following systems are examples of algebras A has the form integrated into GA theory: vector/matrix algebra, complex X A = hAi0 + hAi1 + hAi2 + ··· = hAig, (5) numbers, and quaternions [2], [5], [6], [9]. Such unifying g quality is put into use in this work to expand the capabilities which is comprised of its g-grades (or g-vectors) h·ig, e.g., g = of AFs. 0 (scalars), g = 1 (vectors), g = 2 (bivectors, generated via The fundamentals of GA are provided in the sequel. For an the geometric multiplication of two vectors), g = 3 (trivectors, in-depth discussion, the reader is referred to [2]–[10], [19]– generated via the geometric multiplication of three vectors), [23].