Parallel Gaussian Elimination with Linear Fill
Total Page:16
File Type:pdf, Size:1020Kb
Parallelizing Elimination Orders with Linear Fill Claudson Bornstein Bruce Maggs Gary Miller R Ravi ;; School of Computer Science and Graduate School of Industrial Administration Carnegie Mellon University Pittsburgh PA Abstract pivoting step variable x is eliminated from equa i This paper presents an algorithm for nding parallel tions i i n elimination orders for Gaussian elimination Viewing The system of equations is typically represented as a system of equations as a graph the algorithm can be a matrix and as the pivots are p erformed some entries applied directly to interval graphs and chordal graphs in the matrix that were originally zero may b ecome For general graphs the algorithm can be used to paral nonzero The number of new nonzeros pro duced in lelize the order produced by some other heuristic such solving the system is called the l l Among the many as minimum degree In this case the algorithm is ap dierent orders of the variables one is typically chosen plied to the chordal completion that the heuristic gen so as to minimize the ll Minimizing the ll is desir erates from the input graph In general the input to able b ecause it limits the amount of storage needed to the algorithm is a chordal graph G with n nodes and solve the problem and also b ecause the ll is strongly m edges The algorithm produces an order with height correlated with the total number of op erations work p erformed at most O log n times optimal l l at most O m and work at most O W G where W G is the Gaussian elimination can also b e viewed as an algo minimum possible work over al l elimination orders for rithm that is p erformed on the graph whose adjacency G Experimental results show that when applied after matrix is the matrix representing the system of equa some other heuristic the increase in work and l l is tions Pivoting on a variable corresp onds to usual ly smal l In some instances the algorithm obtains removing a vertex from the graph and forming a clique an order that is actual ly better in terms of work and of its neighbors The number of new edges added to l l than the original one We also present an algo the graph in this pro cess constitutes the ll rithm that produces an order with a factor of log n less Throughout this pap er we assume that our matrices p are symmetric p ositive denite so that our graphs are log n more l l height but with a factor of O undirected our pivots are always nonzero and we can Introduction ignore the issue of numerical stability One of the most p opular metho ds for solving a sys Heuristics for sparse Gaussian elimi tem of linear equations is Gaussian elimination The nation crux of this metho d is to pivot on the variables of the A number of heuristics for minimizing ll for sparse system oneatatime according to some order For ex matrices are available the most p opular b eing nested ample supp ose that the variables x x are to b e n dissection and minimumdegree eliminated according to an order Then in the ith Nested dissection as the name suggests is a recur 1 Emailcfbcscmuedu sive elimination pro cedure It identies a balanced 2 Bruce Maggs is supp orted in part by the Air Force Materiel separator in the graph and sets the no des in the sepa Command AFMC and ARPA under Contract FC rator apart for elimination at the very end The com by ARPA Contracts F and N p onents resulting from removing the separator are re and by an NSF National Young Investigator Award No cursively ordered one after the other and placed b e CCR with matching funds provided by NEC Research fore the separator in the elimination order George Institute and Sun Microsystems Emailbmmcscmuedu rst prop osed this metho d for eliminating no des 3 Gary Miller is supp orted in part by NSF Grant CCR in a mesh and later generalized it in a pap er with Liu and ARPA under Contract N for eliminating the no des in an arbitrary graph Emailglmillercscmuedu Bounds on the ll induced by nested dissection orders 4 R Ravi is supp orted in part by an NSF CAREER Award are known for planar graphs and arbitrary graphs with CCR Emailraviandrewcmuedu b ounded degree The views and conclusions contained here are those of the The minimumdegree heuristic rep eatedly nds a authors and should not b e interpreted as necessarily represent vertex of minimum degree and eliminates it This ing the ocial p olicies or endorsements either express or im plied of AFMC ARPA CMU or the US Government heuristic originated with the work of Markowitz in the late s and has undergone several enhancements in ing the space overhead minimal corresp onds to nd the years since Its p opularity is attested to by its ing an order that has simultaneously low height and inclusion in various publicly available co des such as low ll Gilb ert conjectured the existence of a par MA YALESMP and SPARSPAK In contrast to allel elimination order that has the minimum p ossible nested dissection no p erformance guarantee is known height among all orders and ll that is only a constant for the ll induced by this heuristic In fact there ex factor more than the number of edges in a minimum ist graphs for which the ll induced by the minimum ll order see The hop e was that a small increase degree order can b e very high in ll could b e traded for faster parallel solutions Asp vall disproved this conjecture however by exhibit Recently some hybrid algorithms have b een ex ing a graph for which any order that has the minimum p erimentally shown to pro duce orders that compare p ossible height requires a p olynomial factor more ll favorably with those pro duced by either minimum than the minimum p ossible degree or nested dissection alone Hendrickson and Rothberg and indep endently Liu and Ashcraft Given an interval graph a sub class of chordal prop osed algorithms that rst nd a separa graphs with n vertices and m edges can we nd an tor that partitions the graph into small comp onents order with O m ll and height close to the minimum The minimumdegree heuristic is then used to order p ossible In particular do es a nested dissection order the vertices within each comp onent and also within accomplish this We show that the classical nested the separators In practice b oth algorithms pro dissection pro cedure applied to interval graphs pro p duce orders that compare favorably with stateofthe log n m ll and with height duces an order with O art minimumdegree and nesteddissection algorithms at most O log n times the optimum for that graph In but no b ounds on the amount of ll that they intro fact the b ound on ll is tight if the nested dissection duce are known algorithm is forced to choose a balanced separa tor thus providing a negative answer to our question Our results Even in this very restricted class of graphs nested dis In this pap er we fo cus on parallel elimination or section may generate undesirable ll ders for chordal graphs Chordal graphs are a natural On the p ositive side we show that an extension choice b ecause they are rich in structure and b ecause of nested dissection provides go o d orders We ex they are intimately related to Gaussian elimination or hibit an interval graph algorithm that pro duces orders ders in fact chordal graphs are exactly the class of with O m ll and height within a factor of O log n graphs that have zeroll elimination orders More times the optimum This same algorithm can then over since any elimination order constructs a chordal b e generalized to chordal graphs and pro duces orders completion of a graph that is adds edges to the graph that also have O m ll while having height within so as to make it chordal we can apply our algorithm to an O log n factor of the minimum p ossible While the chordal completion induced by any go o d ordering this guarantee is worse in terms of height than the algorithm one nested dissection provides it is signicantly b et Chordal graphs already have zeroll elimination ter than that given by either a p erfect elimination or orders so how can we p ossibly improve on that We der or by the minimumdegree heuristic In addition prop ose that some extra ll might b e tolerable if paral to the balanced separators used in nested dissection lelism can b e exp osed Although we have thus far de we utilize another kind of separator that we call sen scrib ed Gaussian elimination as if vertices were elim tinels Sentinels help lo calize the ll induced by our inated oneatatime in fact a set of vertices can b e orders without increasing the height by much In ad eliminated in parallel if they are indep endent ie no dition to the b ound on the ll we also show that the two vertices in the set are neighbors Thus in a par total work as well as the front size of our orders are allel elimination order we allow indep endent sets to within a constant factor of the minimum p ossible b e eliminated in one step and we dene the height to Preliminary exp eriments show that the overheads b e the total number of steps A lower b ound on the in ll and height are much b etter than predicted by height of any elimination order is the size of the maxi our theoretical analysis For instance we observed mum clique in the graph since the vertices in a clique the following interesting b ehavior in twodimensional cannot b e eliminated in parallel grids Minimumdegree p erforms b etter than nested Although a strictly sequential order has height n dissection in terms of ll and work on grids with high it is often p ossible to exp ose some parallelism in a asp ect ratio In this case however the orders sequential order The idea is to view a sequential