Index Transformation Algorithms in a Linear Algebra Framework

Index Transformation Algorithms in a Linear Algebra Framework y Alan Edelman Department of Mathematics Massachusetts Institute of Technology Cambridge MA edelmanmathmitedu Steve Heller z S Lennart Johnsson Thinking Machines Corp oration First Street Cambridge MA hellerthinkcom johnssonthinkcom April Abstract We present a linear algebraic formulation for a class of index transformations such as Gray co de enco ding and deco ding matrix transp ose bit reversal vector reversal shues and other index or dimension p ermutations This formulation unies simplies and can b e used to derive algorithms for hyp ercub e multipro cessors We show how all the widely known prop erties of Gray co des and some not so wellknown prop erties as well can b e derived using this framework Using this framework we relate hyp ercub e communications algorithms to GaussJordan elimination on a matrix of s and s Keywords and phrases binarycomplementp ermute binary hyp ercub e Connection Ma chine Gray co de index transformation multipro cessor communication routing shue Simultaneousl y app ears as Thinking Machines technical rep ort TMC y Supp orted by the Applied Mathematical Sciences subprogram of the Oce of Energy Research US Department of Energy under Contract DEACSF z Also aliated with the Division of Applied Sciences Harvard University Intro duction We present a theory for a class of index transformation algorithms that should b e prop erly thought of as a matrixvector pro duct though they rarely are This class is strictly a sup erset of the class known as BCP bitp ermutecomplement In spirit this theory is linked with the ideas in Van Loans new b o ok particularly the notion that matrix factorizations can dene algorithms The principal idea is not the discussion of matrix factorization algorithms p er se The idea is a dierent way of viewing and generating algorithms Van Loan covers computational frameworks for the Fast Fourier Transform Despite dier ences in our approach on this quote we rmly agree The prop er way to discuss a matrixvector pro duct such as the discrete Fourier transform is with matrixvector notation not with vectors of subscripts and multiple summations We should b e as rep elled by scalar notation as we are by assembly language co ding for b oth retard algorithmic development Although it has always b een clear that BCP and larger classes of communications problems can b e formulated as matrixvector pro ducts they rarely have b een Keohane and Stearns address a similar class of p ermutations but do not formulate the problem as a matrixvector pro duct A notable exception is the contemp oraneous work of Cormen for p ermuting data on disk arrays Our motivation stems from communications algorithms for real applications on hyp ercub e mul tipro cessors such as the Connection Machine mo del CM multipro cessor though we b elieve these ideas to have wider applicability Our matrices only contain s and s they describ e transforma n tions on a vector of length indirectly through binary enco dings The most familiar example is bit reversal an op eration used in conjunction with FFTs Bit reversal is a p ermutation of a vector n of length induced by a p ermutation on n ob jects the n bits of the vectors indices One can n n represent this transformation as a p ermutation matrix on the comp onents of the vector For our purp oses it is more convenient to consider the more compact representation of the n n matrix describing the index transformation which in the bit reversal case has s on the northeastsouthwest diagonal and is otherwise Also familiar are socalled dimension transforma tions or index p ermutations These are arbitrary p ermutations of the n bit indices which induce n n p ermutations on elements Why use matrices of order when matrices of order n suce We dene a linear index transformation by i Ai where i is a bit vector with n comp onents A is an n n matrix and the matrixvector multiply is p erformed mo dulo So long as A is nonsingular this n n matrix induces a p ermutation on the n indices Dimension p ermutations are trivial examples of such transformations other examples include Gray co de enco ding and deco ding of arbitrary axes Many real applications on hyp ercub e multipro cessors require complicated comp ositions of these transformations We show that this is not a matter of notation but rather that the existence of a certain kind of convenient algorithm on a hyp ercub e to p erform the data movement given by a linear index transformation is equivalent to the ability to p erform GaussJordan elimination on A without pivoting This ability in turn is related to a familiar condition on the principal submatrices of A Thus the complicated combinatorial problem of devising an algorithm is reduced to the algebraic problem of decomp osing a matrix We b elieve that this is the rst time that the existence of a hyp ercub e communications algorithm has b een related to the ability to p erform GaussJordan elimination In Section we x notation that will b e useful throughout the pap er while Section contains our main results In Section we apply these results towards the sp ecial case of Gray co de enco ding and deco ding while Section considers dimension p ermutations We conclude in Section Notation Let F b e the eld of elements f g with addition and multiplication dened mo dulo In 2 this pap er addition and multiplication are always p erformed mo dulo n We denote the vector space of nvectors with elements in F as F Similarly the set of m n 2 2 mn matrices with elements in F is denoted by F For clarity we sometimes display such matrices 2 2 with empty spaces where the entries are We sometimes consider i or its binary enco ding as the no de address of a hyp ercub e in the usual manner n n Any integer i such that i can b e identied with an element of F by the use of 2 P n1 k i then we identify i with the vector the binary enco ding of the numb er Thus if i k k =0 n T i i Notice that the vector is written with the least signicant bit rst Of course F 0 n1 2 n+1 by app ending an extra zero can b e naturally included as a subset of F 2 We admit that this vector notation for the binary representation of a numb er seems to clash with the usual representation i i i in that the order app ears backwards but the denition as n1 1 0 presented is appropriate and consistent for matrixvector notation We have resisted the temptation mn n to refer to the rst row of a matrix in F or the rst comp onent of a vector in F as the th but 2 2 rather chose the more familiar index origin of one n1 n Some useful vectors are e in which only the nth comp onent is and j in n n k which only the rst n comp onents are These vectors can b e thought of as elements of F for any 2 k n using the natural emb edding Also we can avoid diculties by letting e j 0 0 If x x is any ordered sequence of numb ers then its reversal is the sequence x x 1 k k 1 Linear and Ane Index Transformations We now dene the transformations of interest to us which we refer to as ane or linear Denition An index transformation is dened to be ane if the data in node i is sent to node f i where f i Ai b Cormen calls this class of transformations BMMC for bitmatrixmultiplycomplement Denition An index transformation is dened to be linear if the data in node i is sent to node f i where f i Ai Thus a linear index transformation is an ane transformation that xes the data in node The simplest hyp ercub e communication is the unconditional exchange of data across a xed dimension Algebraically this can b e describ ed by f i i e Another simple hyp ercub e commu k nication sends data to the opp osite corner of the hyp ercub e This is f i i j which describ es n vector reversal Another example of a linear index transformation is a dimension p ermutation considered by such authors as Stone Fraser Nassimi and Sahni Flanders Johnsson and Ho Stout and Wagar and Swarztraub er A dimension p ermutation is dened as the map T f i P i where P is a p ermutation matrix Since p ermutation matrices are orthogonal PP I 2 if it is also symmetric then it is a square ro ot of the identity P I Thus a symmetric p ermutation matrix corresp onds to a disjoint set of dimension pairs b eing exchanged On the other hand circulant p ermutation matrices corresp ond to relab eling dimensions in a way that preserves the circular order of the indices The shue and unshue op erations give two such matrices Circulant p ermutation matrices form a subset of the irreducible p ermutation matrices A matrix A is said to b e irreducible if it has no nontrivial invariant subspaces The irreducible p ermutation matrices corresp ond to the dimension exchange represented by a cycle In Section we will consider the example of Gray co de enco ding and deco ding The basic theorems of algebra tell us that if f i Ai b where A is nonsingular then the map is onetoone Otherwise if the rank of A is r then A maps the hyp ercub e to an r dimensional nr sub cub e This map sends the data in no des to one Denition A conditional exchange across dimension k denoted E is a communication k pattern dened by f i Ai where A is any matrix whose diagonal consists of s and whose odiagonal may possibly be only in the k th row An example of a conditional exchange across dimension is represented by the matrix C B C B E C B 3 A The mapping f i E i describ es a conditional change of the third bit dep ending on the rst 3 and fourth bits We will extend our

Load more