Parallel algorithms based on expander graphs for optical computing Ramamohan Paturi, Dau-Tsuong Lu, Joseph E. Ford, Sadik C. Esener, and Sing H. Lee We consider the task of interconnecting processors to realize efficient parallel algorithms. We propose interconnecting processors using certain graphs called expander graphs, which can provide fast communica- tion from any group of processors to the rest of the network. We show that these interconnections would result in a number of efficient parallel algorithms for sorting, routing, associative memory, and fault-tolerance networks. As the interconnections based on expander graphs are global and irregular, we reason that optical interconnections are preferred to electronic and propose implementation of these interconnections using the programmable optoelectronic multiprocessor architecture. Key words: Optical interconnection, optical computing, expander graphs. 1. Introduction To this end, we consider the design and the construc- To cope with the ever increasing demand on com- tion of expander graphs. We describe a probabilistic power, it is not enough to rely on faster device approach to construct and evaluate good expander puting that ex- technology. It is necessary to utilize parallel process- graphs. We then try to convince the reader ing. Assuming that the communication overhead is pander graphs can indeed result in efficient algorithms discuss an optoe- small and that the algorithm can be fully parallelized, a in a variety of situations. We then task requiring T sequential time steps can be per- lectronic implementation of these interconnection by distributing the task among p networks which combines the optical interconnection formed in Tip steps (VLSI) processors. These two assumptions are the most im- technology with very large scale integration portant considerations in designing efficient parallel technology," thus overcoming the difficulties encoun- consider the design and implementa- tered with pure'VLSI technology. algorithms. We of of such algorithms. More specifically, we investi- In Sec. II we explain the definition and theory tion we present a gate the interconnection properties of very large scale expander graphs. Using this theory, processor networks necessary to support efficient par- probabilistic approach to the construction of expander graphs give allel algorithms. We find certain interconnection net- graphs. In Sec. III we show that expander expander graphs very useful for this pur- rise to efficient parallel algorithms in a number of works called we explain how pose. Interconnections based on expander graphs can application domains. In particular, in constant time. This expander graphs can be used to construct approximate achieve global communication sorting property of expander graphs is successfully exploited halvers, the basic building block of an optimal design of several efficient parallel algorithms.'-10 algorithm. We show how expanders can result in a in the describe However, it has been unclear how to construct and lower delay in routing applications. We also implement good expander graphs. applications in associative memory, object distribu- that the interconnection net- tion, fault-tolerant networks, and error correcting We take the position imple- on expander graphs are the key to imple- codes. In Sec. IV, we discuss approaches to works based and propose an op- menting significantly efficient parallel algorithms. menting irregular interconnections toelectronic system. Finally, in Sec. V, we discuss our conclusions and suggest future research directions. 11. Expander Graphs All authors are with University of California, San Diego, La Jolla, Efficient parallel algorithms rely on the fast transfer California 92093; R. Paturi is in the Department of Computer Sci- within the processor network.12 Al- of of information ence & Engineering, the other authors are in the Department interconnected crossbar network can Electrical & Computer Engineering. though a fully Received 3 January 1990. accomplish such communication in a single time step, 0003-6935/91/080917-11$05.00/0. the number and length of interconnections required 2 © 1991 Optical Society of America. (n for n nodes) make the implementation of such large 10 March 1991 / Vol. 30, No. 8 / APPLIED OPTICS 917 1log n property is indispensable for designing optimal algo- rithms.' 2 One of these is a 0(logn) routing algorithm which can be used as a basis for a general purpose parallel computer.2 A. Definition of Expander Graphs Expander graphs are defined in terms of their prop- erties. For convenience, we describe them as bipartite \ ncreasing ammng/ graphs, i.e., graphs with connections between two dis- crete regions. A bipartite graph G = (I,O,E) has a set I of input nodes and a set 0 of output nodes with E as the set of edges between input and output nodes. We logn consider only those bipartite graphs for which III = 101 and define that III = 101 = n. Edge (ij) E connects Fig. 1. Communication distance on a hypercube with n nodes. input node i with the output node j. For any subset A The nodes are denoted by logn-digit binary addresses. For n = 8, of inputs, we define the neighborhood (A) = U E the longest communication distance is between (000) and (111). If O(ij) , E for some i e information needs to be transmitted from the group of nodes in the A}. The same applies to any 1 subset of outputs shaded region to the remaining nodes, it requires /2 logn - c(logn) /2 with its neighborhood in 0. We also steps, which grows with logn. define a bipartite graph G to be d-regular if the degree, i.e., fanout, of every node in the graph G equals d. For 0 < e < 1 and # > 1, a d-regular bipartite graph G scale networks prohibitively expensive. Often we are = (I,O,E) is called a (d,e,f3) expander if, for all A c I(O) limited to networks with a smaller number of intercon- so that AI < en, the neighborhood (A) of A in G is nections per processor. An example of such a network such that Ir(A)l 2 IAI. In other words, a graph ex- is hypercube with log2 n interconnections per proces- pands if every subset of nodes up to a given size has a sor. Two processors are connected if the Hamming large neighborhood. We call /3the exapnsion factor of distance between their log2 n-bit addresses is 1. The the graph. Typically, we use expander graphs with 3 hypercube and its variants, such as shuffle exchange = (1-e)/e. In the following subsections, we look at the and cube connected cycles, have become popular be- existence and the construction of expander graphs in cause of their relative ease of implementation and greater detail. because a number of algorithms have been implement- ed on them with satisfactory performance. Such net- B. Properties of Expander Graphs works cannot give us optimal parallel algorithms, how- In an expander graph, the size of the neighborhood ever, since they lack sufficient connectivity to of every set is larger than that set by a constant factor. facilitate fast parallel communication. For example, This expanding property gives rise to a number of consider a communication task in which some small interesting computation and communication proper- group, say 10%, of the processors hve information ties. For example, this expansion property gives rise which they need to transmit to the entire network. to approximate halving in a constant number of steps Assuming we have no control over the initial informa- using compare and exchange operations. It also offers tion distribution among the processors, any group of the means of realizing a trade-off between storage ca- 10% of the processors can be considered. Clearly, a pacity for the interconnection patterns and the degree crossbar network can accomplish this task in one step of the network while retaining the error correction, but is technologically expensive. A hypercube can exponential convergence, and robustness properties in also accomplish this task but requires 0(logn) steps, associative memory. Furthermore, this property also which scales with the size of the network (Fig. 1). We ensures multiple paths between the processing ele- need a network which can interconnect an arbitrary ments, providing the necessary redundancy for fault- number of processors in a constant number of steps. tolerant communication networks. These properties Interconnection networks based on expander graphs together with applications are presented in further can provide the solution. They can accomplish this detail in Sec. III. task in a number of steps dependent only on the frac- The large neighborhood criterion is a strong require- tional size of the group and the graph's expansion ment. Consequently, even the proof of existence of factor. The number of steps is independent of net- such graphs is nontrivial. Such a proof is in fact pro- work size. We call this property global communica- vided by a nonconstructive (probabilistic) argument. tion with a constant number of steps. It has been In this argument, we look at the set of all d-regular shown that for any given expansion there exist expan- bipartite graphs with each side having n nodes togeth- der graphs with a constant number of fanouts per node er with a uniform distribution 3 on it. We then show (see Theorem 1 of Sec. II.B).1 This global communi- that the fraction of these graphs that fails to have the cation ability is of fundamental importance for inter- given expanding property is <1. To make this frac- connection networks because a number of optimal al- tion small, we select a suitable d as a function of the gorithms can be developed based on such networks. expansion. Since the fraction of graphs which does In fact, there is some theoretical evidence that this not have the required expanding property is <1, cer- 918 APPLIED OPTICS / Vol. 30, No. 8 / 10 March 1991 tain graphs of degree d must exist that meet the given for estimating the expansion.