arXiv:1407.0961v1 [cs.DS] 3 Jul 2014 • euercrieyadhv aaiiyo sorting of capability pro- These a merge list. have 2-way sorted and list, recursively a employ bitonic cedure obtain algorithms a to merging as converted 2- 2-way arranged then algorithm first popular is merging are Another which lists bitonic 2. sorted is Two to [4]. algorithm limited by merging not merge sublists way modulo multiple odd-even intro- into a lists the was input with sorting of sorted one two merge generalization the into modulo dividing a lists) a as even [3], and duced In (odd list. lists [2] sorted merging sorted odd-even two called merges algorithm for merging procedure popular list. key 2-way One sorted approach. the sorting single is decomposition-based a operation the as merging merged the then Hence, smaller list. even are of which sorted decomposed merging sublists, be and single sorting then a the can into sublists as recursively of merged process Then, sorting are simultaneously. The sublists sublist sorted each sorts the and values) of S fvle)it multiple into input values) the divides it of First, [2]. sorting is steps the two algorithm performs in sorting which popular algorithm, most merge-sort the called hard- of decreasing One and cost. speed ware hardware researchers increasing of to attention rates due attracted throughput have networks parallel [2]. high sorting These simultaneously. [1], achieve operations applications performing can by of variety networks wide Sorting a of performance I 1 egSi hya a,adMgaa ahaewt h Depart- the with are E-mails: USA. Wagh 18015, PA Meghanad University, mdw0 Lehigh and ECE, Yan, of Zhiyuan ment Shi, Feng n ec t fcec ral fet h overall the affects greatly efficiency its hence and O m nti ok efis rps e utwymrigalgorith merging multiway new a propose first we work, this In -a rmliwymrigagrtm sn -otr sba as 2-sorters using algorithms merging multi-way or 2-way Abstract stebscbidn lcss st ral euetenumbe the reduce greatly to as an so algorithm, blocks columnsort building basic the as opr h opeiyi em fterqie ubro gat of number required the of terms in complexity the compare ne Terms Index SS-Mk. O loih stesm steS-k o ierne of ranges wide for SS-Mk, the as same the is algorithm iaysrigntok hr h ai otri implemen is sorter basic the where network, sorting binary a tn soeipratoeaini aaprocessing, data in operation important one is rting NTRODUCTION ( } N ausec in each values @lehigh.edu. log nEhne utwySrigNetwork Sorting Multiway Enhanced An 2 Mrigbsdsrigntok r nipratfml of family important an are networks sorting —Merging-based N ) Mlia,srig merging sorting, —Multiway, ai otr osort to sorters basic + 1 ⌈ m/ n 2 wymrigagrtm eerdt sS-k htuses that SS-Mk, as to referred algorithm, merging -way sublists ⌉ tgs( stages N egSi hya a,adMgaa Wagh Meghanad and Yan, Zhiyuan Shi, Feng nus hl h smttccmlxt i em ftereq the of terms (in complexity asymptotic the While inputs. ae on Based n asalrsequence smaller (a ≤ m .Bsdo u egn loih,w lopooeasriga sorting a propose also we algorithm, merging our on Based ). list asequence (a { e29 yan, fes209, N u loih eursfwrsresta h SM.Final SS-Mk. the than sorters fewer requires algorithm our , i uligbok.A lentv st use to is alternative An blocks. building sic e ntrsodlgcadsae ierywt h ubrof number the with linearly scales and logic threshold in ted N with m s o ierne of ranges wide For es. fsresa ela h aec.Bsdo oie Leighto modified a on Based latency. the as well as sorters of r ✦ n SM,bsdo h oie ounotwsproposed was columnsort modified the on based SS-Mk, lc ntee2wymrigagrtmi 2-sorter, a is algorithm building a merging simply basic 2-way is the which these However, [6]. in in block proposed sorting are low-latency units high-throughput of a design the Recently, impractical. modular in it constant makes large which very expression, depth a is there However, proposed. oee,a2wymrei tl sdfrtemerging the for an used [14], still is In merge process. 2-way a However, from with constructed proposed networks was sorting algorithm columnsort In modified re- provided. a no was [8], network and sorting specific columns sorted of no construction sorting single cursive columnsort, for a original provided the obtain was In operation to order. row columns in sorted list all merges since columnsort, it named and merge-sort odd-even the of san as 1(a). Fig. in shown as ae K otn ewr,with network, sorting AKS named otr sbscbidn lcs egtnpooe an proposed Leighton sorting blocks. for building algorithm basic as sorters the in operation Nakatani by bitonic the of greatly. where reduced [7], be Batcher to and expected sorters is of latency An number the the as blocks, well building as basic as used are ausin values twsas oiae yteueof use the n by [7]. motivated algorithm also merging was general- odd-even It a Batcher’s as the proposed of first ization was This blocks. building sic n ( nta fuig2-sorters, using of Instead sresa ai uligbok htmerges that blocks building basic as -sorters otn ewrs otmresrigntok r ae on based are networks sorting merge Most networks. sorting -Sorters n n ≥ wymrigagrtmwsfis rpsdb Lee by proposed first was algorithm merging -way r × 2 ausi nttm 8,[] ic ag sorters large Since [9]. [8], time unit in values ) c O arx[2.Ti loih sageneralization a is algorithm This [12]. matrix (log n sresa ai uligbok a proposed. was blocks building basic as -sorters tal. et N n 2 u loih eursfwrgtsta the than gates fewer requires algorithm our , wymrigagrtmwsproposed was algorithm merging -way N n 2 wymrigagrtm tl s 2- use still algorithms merging -way ) × 1] 1] oee,tecombining the However, [11]. [10], tgs[] n[] otn network, sorting a [5], In [2]. stages n ie ubro otr)o u sorting our of sorters) of number uired r 2 wymrigagrtm named algorithm, merging -way it of lists wthn lmn rcomparator or element switching n sntrsrce o2 version A 2. to restricted not is n sres nta f2-sorters, of instead -sorters, n srescnb sda ba- as used be can -sorters c ausec,represented each, values grtm hc requires which lgorithm, n n O sres( -sorters sres hc sort which -sorters, (log n y econsider we ly, otdlssof lists sorted N nus and inputs, ) n tgswas stages ≥ n’s 2 [13]. ) 1 2 with n-sorters as basic building blocks, where n is prime. closed-form expression of the number of sorters for For n sorted lists of m values each, the idea is to sort the the SS-Mk [14], since it was not provided in [14]. m × n values first in each row and then in slope lines Then we present extensive comparisons between the with decreasing slope rates. An improved version of the latency and the number of sorters required by our SS-Mk merge sort, called ISS-Mk, was provided in [15], approach and the SS-Mk [14]. where n can be any integer. We compare our sorting • Finally, we show an implementation of a binary scheme with the SS-Mk but not the ISS-Mk, because for sorting network in threshold logic. With an imple- our interested ranges of N, the ISS-Mk requires larger mentation of a large sorter in threshold logic, we latency due to a large constant. compare the performance of sorting networks in In this work, we propose an n-way merging algorithm, terms of the number of gates. which generalizes the odd-even merge by using n-sorters The rest of the paper is organized as following. In as basic building blocks, where n (≥ 2) is prime. Based Sec. 2, we briefly review the background of sorting on this merging algorithm, we also propose a sorting networks. In Sec. 3, we propose a multiway merging p p(p−1) algorithm. For N = n input values, p + ⌈n/2⌉× 2 algorithm with n-sorters as basic blocks. In Sec. 4, we stages are needed. The complexity of the sorting network introduce a multiway sorting algorithm based on the is evaluated by the total number of n-sorters. The closed- proposed merging algorithm, and show extensive results form expression for the number of sorters is also derived. for the comparison of our sorting algorithm and previ- Instead of 2-sorters, n-sorters (n > 2) are used as ous works. In Sec. 5, we focus on a binary sorting net- basic blocks in this work. This is because larger sorters work, where basic sorters are implemented by threshold have some efficient implementation. For example, for logic and have complexity linear with the input size, and binary sorting in threshold logic, the area of an n-sorter measure the complexity in terms of number of gates. scales linearly with the number of inputs n, while the Finally Sec. 6 presents the conclusion of this work. latency stays as a constant. Hence, a smaller number of sorters and latency of the whole sorting network can be 2 BACKGROUND achieved. However, we cannot use arbitrary large sorters as basic blocks, since larger sorters are more complex A sorting network is a feedforward network, which and difficult to be implemented. Hence, the benefit of gives a sorted list for unsorted inputs. It is composed using a larger block diminishes with increasing n. We of two items: switching elements (or comparators) and assume that the size of basic sorter n ≤ 20 and 10 wires. The depth of a comparator is defined to be the when evaluating the number of sorters and latency. Our longest length from the inputs of the sorting network algorithm works for any upper bound on n, and one to that comparator’s outputs. The latency of the sorting can plug any upper bound on n into our algorithm. network is the maximum depth of all comparators. The Asymptotically, the number of sorters required by our network is oblivious in the sense that the time and loca- sorting algorithm is on the same order of O(N log2 N) tion of input and output are fixed ahead of time and not as the SS-Mk [14] for sorting N inputs. Our sorting dependent on the values [2]. We use the Knuth diagram algorithm requires fewer sorters than the SS-Mk in [14] in [1] for easy representation of the sorting networks, in wide ranges of N. For instance, for n ≤ 20, when where switching elements are denoted by connections on N ≤ 1.46 × 104, our algorithm requires up to 46% fewer a a set of wires. The inputs enter at one side and sorted sorters than the SS-Mk. When 1.46×104
Fig. 1. (a) 2-sorter (y1 ≤ y2); (b) n-sorter (y1 ≤ y2 ≤···≤ yn).
Fig. 2. The odd-even merge of two sorted lists of 4 values each using 2-sorters. Fig. 3. Iterative construction rule for the n-way merger [7].
The former is an efficient sorting technique based on the combining network in the merging network in Fig. 3 still divide-and-conquer approach, which decomposes the uses 2-sorters as basic blocks. In [12], Leighton proposed inputs into two sublists (odd and even), sorts each sub- a columnsort algorithm, which showed how to sort an list, and then merges two sorted lists into one. Further m × n matrix denoting the n sorted lists of m values decomposition and merging operations are applied on each. A modification of Leighton’s columnsort algorithm the sublists. An example of odd-even merging network was given in [8]. In [14], [15], merging networks with using 2-sorters is shown in Fig. 2, where two sorted n-sorters as basic blocks are introduced based on the (0) (0) (0) (0) lists, hx1,1, ··· , x1,4i and hx2,1, ··· , x2,4i, are merged as modified Leighton’s columnsort algorithm. (2) (2) (2) (2) a single list hx1,1, ··· , x1,4, x2,1, ··· , x2,4i in two stages. In this work, we focus on multiway merge sort with Instead of merging two lists, multiple sorted lists can binary values as inputs. Our merge sort also works be merged as a single sorted list simultaneously. An n- for arbitrary values, which is justified by the following way merger (n ≥ 2) of size m is a network merging theorem. n sorted lists of size m (m values) each into a single Theorem 2.1 (Zero-one principle [2]). If a network with n sorted list in multiple stages. This was first proposed as input lines sorts all 2n lists of 0s and 1s into nondecreasing a generalization of the Batcher’s odd-even merging algo- order, it will sort any arbitrary list of n values into nonde- rithm. It is also motivated by the use of n-sorters, which creasing order. sort n (n ≥ 2) values in unit time [8], [9]. Since large sorters are used as basic building blocks, the number of ULTIWAY ERGING sorters as well as the latency is expected to be reduced 3 M M greatly. Many multiway merging algorithms exist in the In the following, we propose an n-way merging literature [7], [8], [10]–[17]. The algorithms in [16], [17] algorithm with n-sorters as basic building blocks as implement multiway merge using 2-sorters. In [7], a gen- shown in Alg. 1. We consider a sorting network, where eralization of Batcher’s odd-even is introduced as shown all iterations of Alg. 1 are simultaneously instantiated in Fig. 3, where an n-way merger of n lists of size ud is (loop unrolling). We refer to the instantiation of iteration decomposed into d n-way mergers of n sublists of size i of Alg. 1 as stage i of the sorting network. The sorters u plus a combining network. Each of the small n-way in the last for loop in Alg. 1 consist of the last stage. (0) (0) (0) mergers is further decomposed similarly. However, the Let the n sorted input lists be hxj,1 , xj,2 , ··· xj,mi 4
Algorithm 1 Algorithm for n-way merging network. (0) (0) (0) Input: n sorted lists hxj,1 , xj,2 , ··· xj,mi for j =1, ··· ,n;
i =1; m while i ≤⌈ 2 ⌉ do for j =1 to n − 1 do Apply (m − i)-spaced sorters between lists j and j +1; end for Merge all (m − i)-spaced sorters; (i) (i) (i) Update n sorted lists hxj,1, xj,2, ··· xj,mi for j = 1, ··· ,n; i = i +1; end while for j =1 to n − 1 do Apply (m − 1)-sorters on m − 1 adjacent lines with (i−1) (i−1) first half, xj,m−k, from list j and second half, xj+1,k, m−1 from list j +1, where k =1, ··· , 2 ; end for Fig. 4. The network for sorted lists of wires. Output: Sorted lists. n m
Lemma 3.2. In each stage of Alg. 1, there are at most four for j = 1, ··· ,n. Denote the values of j-th list after cases of adjacent two sorters as shown in Fig. 5. If m is prime, (k) (k) (k) m case IV is impossible. stage k by (xj,1 , xj,2 , ··· , xj,m). After T = 1+ ⌈ 2 ⌉ stages, all input lists are sorted as a single list, (T ) (T ) (T ) (T ) (T ) (T ) (T ) We first show that the first connections of adjacent hx1,1 , x1,2 , ··· , x1,mi, hx2,1 , x2,2 , ··· , x2,mi, ··· , hxn,1 , two sorters, S1 and S2, belong to either the same list or (T ) (T ) xn,2 , ··· , xn,mi. adjacent two lists. The same relation is true for the last For convenience of describing and proving our al- connections of S1 and S2. This gives us a total of four gorithm, we introduce some notations and definitions. cases as shown in Fig. 5, where b ≥ a+1 for Fig. 5(a)-(c), Denote the number of zeros in the j-th list after stage i and b ≥ a for Fig. 5(d) such that S1 and S2 have a size (i) m of at least two. as rj , where i = 1, 2, ··· , ⌈ 2 ⌉ +1 and j = 1, ··· ,n.A sorter is called a k-spaced sorter if its adjacent inputs The following theorem proves the correctness of span k other wires and each connection of the same Alg. 1. sorter comes from different lists of m wires, where Theorem 3.1. For a prime m in Alg. 1, all lists are self- 0 ≤ k ≤ m − 1. For simplicity, we arrange the sorters in sorted after every stage. In particular, all lists are sorted after the order of their first connections in each stage. Denote the final stage. {1, 2, ··· ,m} as Zm. Two k-spaced sorters are said to be adjacent if they connect adjacent two wires, x(i) and The theorem can be proved by induction on i. j,k In Alg. 1, the latency increases linearly with ⌈ m ⌉. x(i) , respectively, for some j ∈ Z and k ∈ Z . 2 j,k+1 m m−1 When m is large, the latency is also very large. By further Then, our n-way merging Alg. 1 can be intuitively decomposing m into a product of small factors, we can understood as flooding lists with zeros in descending reduce the latency significantly. In the following, we order. The correctness of Alg. 1 can be shown by first propose Alg. 2 for merging n lists of m values, where proving the following lemmas. See the appendix for the m = np−1 for p ≥ 2. When m is not a power of n, we proofs of the following lemmas and theorems. ′ can use a larger network of m′ = np > m inputs. For Lemma 3.1. Apply (m − 1)-spaced sorters to n lists of m any q in stage i (2 ≤ i ≤ p − 1), denote the number of (i) values, hxj,1, xj,2, ··· , xj,mi, for j = 1, ··· ,n. The outputs zeros in each new formed list after stage i as rj,q , where of each list are still sorted, hx′ , x′ , ··· , x′ i, for j = i (i) j,1 j,2 j,m j = 1, ··· ,n . Assume two dummy lists with r0,q = n 1, 2, ··· ,n. (i) i and rni+1,q =0 are appended to the two ends of n lists. For n sorted lists of m values, there are m (m − 1)- The correctness of Alg. 2 can be shown by first proving spaced sorters as illustrated in Fig. 4(a). The proof of the the following lemma. lemma can be reduced to showing that any two wires Z Lemma 3.3. In Alg. 2, the new lists in stage i with respect s,s + l ∈ m of each list connected by the s- and (s + l)- to q are self-sorted. The numbers of zeros of all new lists after th sorters are sorted. The simplified network is shown stage i are non-increasing, in Fig. 4(b). Without lose of generality, we can choose (i) (i) i l =1. rj,q ≥ rj+1,q for j =1, ··· ,n − 1, 5
Fig. 5. Adjacent two sorters S1 and S2 in each stage of Alg. 1 can be classified into four four cases. (a) Case I v−w v−1 m−w+1 m (∆= b−a ); (b) Case II (∆= b−a+1 ); (c) Case III (∆= b−a+1 ); (d) Case IV (∆= b−a+2 ).
Algorithm 2 Algorithm for combining n lists of m = np−1 values. (0) (0) (0) Input: n sorted lists hxj,1 , xj,2 , ··· xj,mi for j = 1, ··· ,n and m = np−1; i =1; for q =1 to np−2 do (0) (0) (0) Apply Alg. 1 on hxj,q , xj,np−2+q, xj,2np−2+q, ··· , (0) xj,(n−1)np−2+qi for j = 1, ··· ,n and obtain a single sorted list (1) (1) (1) (1) (1) hx1,q, x1,np−2+q, ··· x1,(n−1)np−2+q, x2,q, x2,np−2+q, ··· , (1) (1) x2,(n−1)np−2+q, ··· , xn,q, (1) (1) xn,np−2+q, ··· , xn,(n−1)np−2+qi; Fig. 6. A 3-way merging network of N = 3 × 7 inputs end for implemented via 7 stages. for i =2 to p − 1 do for q =1 to np−1−i do Group n neighboring values of where i =2, ··· ,p − 1 and q =1, ··· ,np−1−i. Furthermore, (i−1) (i−1) (i−1) (i−1) − − − − − − n hxj,q , xj,np i 1+q, xj,2np i 1+q, ··· xj,(n−1)np i 1+qi there are at most consecutive lists that have between 1 and for j = 1, ··· ,n and denote the new lists n − 1 zeros, (i−1) (i−1) (i−1) as hx , x p−i−1 , ··· x p−i−1 i for (i) (i) (i) j,q j,n +q j,(n−1)n +q r(i) = n > r ≥···≥ r > 0= r for l ≤ n, j =1, ··· ,ni; s,q s+1,q s+l,q s+l+1,q n i for k =2 to ⌈ 2 ⌉ do where s ≥ 0 and s + l ≤ n . Apply (n−k)-spaced sorters between lists j and See Sec. A.4 for the proof. j +1; The following theorem proves the correctness of end for Alg. 2. Apply (n − 1)-sorters between lists j and j +1 for j =1, ··· ,ni − 1; Theorem 3.2. Alg. 2 combines n sorted lists of m = np−1 Obtain a single sorted list values as a single sorted list. (i) (i) (i) (i) hx , x − −1 , ··· , x − −1 , x , 1,q 1,np i +q 1,(n−1)np i +q 2,q 1 + (p − 1)⌈ n ⌉ (i) (i) (i) In Alg. 2, the latency is reduced to 2 for p−1 x2,np−i−1+q, ··· , x2,(n−1)np−i−1+q, ··· , xni,q, n sorted lists of m = n values. (i) (i) In the following, we show two examples for com- xni,np−i−1+q, ··· , xni,(n−1)np−i−1+qi; end for parison of the two algorithms. First, a 3-way merging end for network of N = 3 × 7 inputs via Alg. 1 is shown in Output: Sorted list. Fig. 6. Then, a 3-way merging network of N = 3 × 9 inputs via Alg. 2 is shown in Fig. 7. Though there are more inputs in Fig. 7 than that in Fig. 6, the latency of 6
Mk [15], because for our interested ranges of N, the ISS- Mk requires larger latency due to a large constant.
4.1 Multiway sorting algorithm Based on the multiway merging algorithm in Sec. 3, we proposed a parallel sorting algorithm using a divide- and-conquer method. The idea is to first decompose large list of inputs into smaller sublists, then sort each sublist, and finally merge them into one sorted list. The sorting of each sublist is done by further decomposition. For instance, for N = np inputs, we first divide the np inputs into n lists of np−1 values. Then we sort each of these n lists and combine them with Alg. 2. The sorting operation of each of the n lists is done by dividing the np−1 inputs into n smaller lists of np−2 values. We repeat the above operations until that each of n smaller lists Fig. 7. A 3-way merging network of N = 3 × 9 inputs contains only n values, which can be sorted by a single implemented via 5 stages. n-sorter. The detailed procedures are shown in Alg. 3.
Algorithm 3 Algorithm for sorting N = np values. p (0) (0) (0) Input: N = n values, x1 , x2 , ··· , xnp ; Partition the N = np values as np−1 lists of n values (0) (0) (0) p−1 each, (xj,1 , xj,2 , ··· , xj,n) for j =1, ··· ,n ; Apply one n-sorter on each of np−1 lists and obtain (1) (1) (1) p−1 hxj,1 , xj,2 , ··· , xj,ni for j =1, ··· ,n ; for i =2 to p do for j =1 to np−i do (i−1) (i−1) Apply Alg. 1 on hx(j−1)n+k,1, x(j−1)n+k,2, ··· , (i−1) x(j−1)n+k,ni−1 i for k = 1, ··· ,n, and obtain a (i) (i) (i) single sorted list hxj,1, xj,2, ··· xj,ni i; end for end for Output: Sorted list.
For example, a 3-way sorting network of N = 33 inputs is shown in Fig. 8. The first stage contains 9 3-
3 sorters. The second stage contains 3 three-way mergers Fig. 8. A 3-way sorting network of N = 3 inputs with a depth of 3. The last stage contains a three-way implemented via 9 stages. merger with a depth of 5. The total depth is given by 9.
Alg. 2 is smaller due to recursive decomposition. The 4.2 Latency analysis numbers of sorters in Figs. 6 and 7 are given by 40 and 41, respectively. For six more inputs, it requires only one First, we focus on the latency for sorting N values. The more sorter in Fig. 7. Hence, Alg. 2 can be more efficient latency is defined as the number of basic sorters in the than Alg. 1 for a large m. longest paths from the inputs to the sorted output. In Alg. 3, there are p iterations. In iteration i, there are ni merging networks, each of which is to merge n sorted 4 MULTIWAY SORTING lists of np−i values. For iteration i, the latency is given i−1 n In this section, we first focus on how to construct sorting by Lour(n,n )= 1+(i−1)⌈ 2 ⌉. For a sorting network of networks with n-sorters using the multiway merging N = np values via Alg. 3, by summing up the latencies algorithm in Sec. 3. Then, we analyze the latency and the of all levels, we obtain the total latency number of sorters of the proposed sorting networks by p p i−1 deriving the closed-form expressions. We compare them Lour(n ) = i=1 Lour(n,n ) n p(p−1) (1) with previously proposed SS-Mk in [14] but not the ISS- = pP+ ⌈ 2 ⌉× 2 . 7
The closed-form expression of latency for the SS-Mk of the merging network of n lists of np−i values each is given in [14] is given by
p−i † † † p (p − 1)(p − 2) S (n,n )= M − + K − + C , L (n )=1+(p − 1)n + ⌈log n⌉. (2) SS−Mk np i n,np i n SS−Mk 2 2 p † (n+1−⌈n/2⌉)(n−⌈n/2⌉) (⌈n/2⌉+1)(⌈n/2⌉−2) We compare our latency for sorting N = n values where Mnp−i = 2 + 2 with that for the SS-Mk in [14]. From Eqs. (1) and (2), p−i † p−1−i p−i +2 n K − = ⌈log n ⌉n + (n − p , n,np i 2 for N = n inputs, p should be as small as possible p−1−i ⌈log2 n ⌉+1 † 3)2 , and Cn = (⌈n/2⌉ − 2)n − to obtain small latencies. In Table 1, we compare the 3(⌈n/2+1⌉)(⌈n/2⌉−2) (n+1−⌈n/2⌉)(n−⌈n/2⌉) latencies of Eqs. (1) and (2) for small p (p =2, 3, 4). It is 2 − 2 − (n − 3). easily seen that our implementation has a smaller latency The total number of sorters of the sorting network via than the SS-Mk in [14] for a prime greater than 3. It is the SS-Mk in [14] is given by p p also observed that Lour(2 ) = LSS−Mk(2 ) = p(p + 1)/2 p p−1 i−1 p−i p−1 TSS−Mk(n ) = n · SSS−Mk(n,n )+ n i=1 − for n =2, which is the same as the odd-even merge sort † np 1−1 † p−1 = (Pp − 1) · M −1 + · C + n in [2]. np n−1 n p−1 p−2 +n i=1 ⌈i log2 n⌉ p−1 TABLE 1 i−1 ⌈(p−1−i) log2 n⌉+1 + i=1Pn (n − 3)2 , p Comparison of latencies of sorting networks of N = n P (4) p inputs via the SS-Mk in [14] and our implementation. As N → ∞, TSS−Mk(n ) is on the order N(log N−log n) N log N(log N−log n) of O(B1 (log n)/n + B2 n log n + p = 2 p = 3 p = 4 N(log N−log n) N B3 n log n + B4 n ). [14] 1+ n 1 + 2n + ⌈log2 n⌉ 1 + 3n + 3⌈log2 n⌉ p n n n According to the big-O expressions of Tour(n ) and Ours 2+ ⌈ 2 ⌉ 3 + 3⌈ 2 ⌉ 4 + 6⌈ 2 ⌉ p TSS−Mk(n ), when n is bounded, the asymptotic bounds on the number of sorters required by both our Alg. 3 and the SS-Mk in [14] are given by O(N log2 N), which is also the asymptotical bound for the odd-even and 4.3 Analysis of the number of sorters bitonic sorting algorithms [2], [4]. When N is fixed and In the following, we compare the number of sorters n increases, the first term of the big-O expressions of p p of our algorithms with the SS-Mk in [14]. Since the Tour(n ) and TSS−Mk(n ) decreases first, then increases, distribution of sorters for an arbitrary sorting network and decreases to zero when n → N. While other of N inputs is not known, we assume that any m-sorter terms decrease monotonically with n. Hence, if n is p (m p−i work, we assume that the basic sorter size is limited. For p−i ∗ n − 1 ∗ p−i Sour(n,n ) = (p − i) · M p−i + · C + n , a given N, we take the total number of sorters in Eqs. (3) n n 1/p n − 1 and (4) as a function of p with n = N ≤ nb, where ∗ ⌈n/2⌉(⌈n/2⌉−1) p−i ∗ nb is the upper bound of the basic sorter size. When N where Mnp−i = 1+ 2 n and Cn = is not a power of a prime, we append redundant inputs 3⌈n/2⌉(⌈n/2⌉−1) ′ ′ (⌈n/2⌉− 1)n − 2 − 1. By summing up the of 0’s and get a larger N such that N is a power of a numbers of sorters of all mergers in all stages, we obtain prime. Hence, we have n′ = N ′1/p = ⌈⌈N 1/p⌉⌉, where the total number of sorters, which is given by ⌈⌈x⌉⌉ denotes the smallest prime larger than or equal to p p−1 i−1 p−i p−1 x. There exists an optimal p such that the total number Tour(n ) = i=1 n · Sour(n,n )+ n p−1 − of sorters is the minimum. We search for the optimal p’s Pp(p−1) ∗ (p−1)n np 1−1 = 2 · Mnp−1 + n−1 − (n−1)2 for our Alg. 3 and the SS-Mk [14] using MATLAB. By ∗ p−1 h i ·Cn + pn , plugging the optimal p’s into Eqs. (3) and (4), we obtain (3) the total number of sorters for sorting networks of N p As N → ∞, Tour(n ) is on the order of inputs. N log N(log N−log n) N(log N−log n) N log N O(A1 (log n)2/n + A2 log n + A3 n log n ). We compare the number of sorters for sorting net- Similarly for the SS-Mk in [14], the number of sorters works via the Batcher’s odd-even algorithm [2], our 8 TABLE 2 Alg. 3, and the SS-Mk [14] for wide ranges of N. The Comparison of the number of sorters for sorting N =2k results are show in Fig. 9. The numbers of sorters are inputs (1 ≤ k ≤ 16) with n ≤ 20 via the SS-Mk in [14] and illustrated by staircase curves, because we use a larger our Alg. 3. sorting network for N not being a power of prime. From Fig. 9, the Batcher’s odd-even algorithm using 2-sorters always requires more sorters than both our Alg. 3 and N SS-Mk Ours Rd. (%) the SS-Mk in [14]. For both our Alg. 3 and the SS- 2 1 1 0.0 4 5 5 0.0 Mk [14], the number of sorters is smaller for a larger 8 11 11 0.0 nb, meaning that using larger basic sorters reduces the 16 38 30 21.05 number of sorters. For the comparison of the number 32 95 65 31.58 64 347 207 40.35 of sorters required by our Alg. 3 and the SS-Mk [14], 128 566 326 42.40 there are three scenarios with respect to three ranges of 256 1250 690 44.80 2 N. We first focus on nb = 10. For N ≤ 6.25 × 10 , our 512 3952 3500 11.44 Alg. 3 has fewer or the same number of sorters than 1024 8287 6378 23.04 2048 15595 12039 22.80 the SS-Mk as shown in Fig. 9. For some segments in 4096 44652 33891 24.10 6.25 × 102 Latency 60 our Alg. 3 and the SS-Mk [14], the latency can be reduced by having a larger nb. The latency of our Alg. 3 is not 40 4 greater than the SS-Mk for N ≤ 2 × 10 for both nb = 10 and nb = 20 as shown in Fig. 10. This is because our 20 Alg. 3 tends to use large sorters, leading to less stages 0 of sorters. We note that the latency goes up and down 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 for some N in Fig. 10. This is because of the switching N x 104 from a smaller basic sorter to a larger one to reduce the number of sorters. Fig. 10. Comparison of the latency for sorting N inputs with n ≤ 10 and n ≤ 20 via the SS-Mk in [14] and our x 105 Alg. 3. 18 Odd-even (n = 2) 16 SS-Mk (n ≤ 10) To some researchers’ interest, we also compare the 4 Our (n ≤ 10) x 10 14 6 SS-Mk (n ≤ 20) number of sorters for N being a power of two. The Ours (n ≤ 20) results are shown in Table 2, where columns two and 12 4 three show the numbers of sorters for the SS-Mk and our 10 2 Alg. 3, respectively, and column five shows the reduction 8 by our Alg. 3 compared with the SS-Mk [14]. For our 0 # of sorters 0 1000 2000 Alg. 3, there are up to 46% fewer sorters than the SS-Mk 6 in [14] for N =2i, for i =4, 5, ··· , 16. It is also observed 4 that a greater reduction is obtained for small p, meaning 2 our approach is more efficient for networks with larger sorters as basic blocks. 0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 N x 104 5 APPLICATION IN THRESHOLD LOGIC Fig. 9. Comparison of the number of sorters (n ≤ 10 and In Sec. 4.4, we assume all basic sorters in the sorting n ≤ 20) for sorting N inputs via the SS-Mk in [14] and our network are the same and measure the complexity by Alg. 3. the number of sorters, since the distribution of sorters is 9 unknown. This would overestimate the total complexity. In this section, we focus on the threshold logic and measure the complexity by the number of threshold gates. In the following, we first briefly introduce the threshold logic, which is very powerful for computing complex functions, such as parity function, addition, multiplication, and sorting, with significantly reduced number of gates. Then, we present an implementation of a large sorter in threshold logic. Last, we compare the complexity of sorting networks in terms of the number Fig. 12. Sorters implemented in threshold logic (a) 2- of gates. This is a very narrow application in the sense sorter; (b) n-sorter. that sorters are implemented by threshold logic and the inputs are binary values. 5.3 Analysis of number of gates 5.1 Threshold logic In the following, we assume all gates are the same and derive the total number of gates. The sorting network A threshold function [18] f with n inputs (n ≥ 1), of N inputs is composed of multiple stages, of which x1, x2, ··· , xn, is a Boolean function whose output is each partially sorts N values. Not all values in each determined by stage participate the comparison-and-switch operation. A simple way to count the gates is to insert buffer n 1 if i=1 wixi ≥ T gates in each stage to store values without involving f(x1, x2, ··· , xn)= (5) 0 otherwise,P any sorting operation. Buffer insertion is also needed for implementation of threshold logic in some nanotech- where wi is called the weight of xi and T the thresh- nology, where synchronization is required for correction old. In this paper we denote this threshold function operation. Hence, each stage contains N gates and the as [x1, x2, ··· , xn; w1, w2, ··· , wn; T ], and for simplicity total number of gates is obtained by multiplying N to x w x sometimes denote it as f = [ ; ; T ], where = the latency. Note that N does not have to be a power w (x1, x2, ··· , xn) and = (w1, w2, ··· , wn). The physical of n. Hence, the total number of gates of our Alg. 3 and entity realizing a threshold function is called a threshold the SS-Mk [14] are simply given by gate, which can be realized with CMOS or nano tech- nology. Fig. 11 shows the symbol of a threshold gate Qour(N)= N · Lour(N), (6) realizing (5). and QSS−Mk(N)= N · LSS−Mk(N). (7) If n is bounded, the total numbers of gates in Eqs. (6) and (7) have an order of O(N log2 N), which is the same as the order for the numbers of sorters via our Alg. 3 Fig. 11. Threshold gate realizing f(x) for n inputs, and the SS-Mk in [14] in Sec. 4.3. x1, x2, ··· , xn, with corresponding weights ω1,ω2, ··· ,ωn To derive the accurate number of gates, we first and a threshold T . derive the number of buffers added for Eqs. (6) and (7). When N is a power of prime, the number of buffers for sorting N = np values via our Alg. 3 and the SS-Mk [14] can be easily obtained due to 5.2 n-sorter a regular structure. For our Alg. 3, the number of p−2 n2+6n−5 buffers is given by Gour(N) = (p − 1)n 4 + Binary sorters can be easily implemented in threshold p−1 p−2 ((p−2)n −(p−1)n +1)(n+5) (p−1)(p−2) p−1 logic. In [19], a 2-by-2 comparator (2-sorter) was imple- 4(n−1) + 2 n for n 6= 2 mented by two threshold gates as shown in Fig. 12(a). and G(np) = (p2 − p +4)2p−1 − 2 for n =2. Similarly, we Similarly, we introduce a threshold logic implementation derive the number of buffers for the SS-Mk in [14], which p ⌈(i−2) log2 n⌉+1 of an n-sorter as shown in Fig. 12(b), where n threshold is given by GSS−Mk(N) = 2 i=2(2 − p−1 2 2 p−1 p−i (n −1)(n −5) (p−1)(n−1) n gates are required. As shown in Fig. 12, the number 1)n + 2(n−1) + P4 for n 6= 2 and of gates of an n-sorter scales linearly with the number G(np) = (p2 −p+4)2p−1 −2 for n =2. By subtracting the of inputs n. Hence, large sorters are preferred to be number of buffers from Eqs. (6) and (7), we obtain the used as basic blocks. However, larger sorters are more total numbers of gates for our algorithm and the SS-Mk complex and expensive to be implemented. For practical as shown in the following, concerns, such as fan-in and cost, some limit on the size p p p p of basic sorters is assumed. Rour(n )= n · Lour(n ) − Gour(n ), (8) 10 x 106 and 2.5 Odd-even (n = 2) p p p p RSS−Mk(n )= n · LSS−Mk(n ) − GSS−Mk(n ). (9) SS-Mk (n ≤ 10) 4 Our (n ≤ 10) x 10 Though it would overestimate the total number of gates 2 15 SS-Mk (n ≤ 20) Ours (n ≤ 20) by adding buffers. However, the asymptotic gate counts 10 p p are not affected, since both Gour(n ) and GSS−Mk(n ) 1.5 have the same order of O(N log2 N). 5 0 # of gates 1 0 1000 2000 5.4 Comparison of the number of gates In the following, we first compare the number of gates with consideration of buffers. Using the same idea as in 0.5 Sec. 4.3, we search for the optimal p’s of Eqs. (6) and (7) using MATLAB. For n ≤ 10 and n ≤ 20, the numbers 0 of gates of the SS-Mk and our two implementations are 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 N x 104 illustrated in Fig. 13. We also plot the odd-even sorting for comparison. The curves in Fig. 13 are segmented Fig. 13. Comparison of the number of gates (n ≤ 10 and linear lines. This can be explained by Eqs. (6) and (7), n ≤ 20) for sorting N inputs via the SS-Mk in [14] and our which are functions of N and latency. From Fig. 13, the Alg. 3. Batcher’s odd-even algorithm using 2-sorters has more gates than both our algorithm and the SS-Mk in [14]. For both our Alg. 3 and the SS-Mk [14], the number of 140 Odd-even (n = 2) gates is smaller with a larger nb, meaning that using SS-Mk (n ≤ 10) 120 larger basic sorters reduces the number of gates. For Our (n ≤ 10) the comparison of the number of gates required by our SS-Mk (n ≤ 20) n ≤ Alg. 3 and the SS-Mk [14], there are also three scenarios 100 Ours ( 20) with respect to three ranges of N. We first focus on 4 80 nb = 10. For N ≤ 1.68 × 10 , our Alg. 3 has fewer or the same number of gates than the SS-Mk as shown in Latency 60 Fig. 13. For 1.68×104 TABLE 3 TABLE 4 Comparison of the number of gates with buffers for Comparison of the number of gates without buffers for sorting N =2k inputs (1 ≤ k ≤ 16) with n ≤ 20 via the sorting N = np inputs for n ≤ 10 via the SS-Mk in [14] SS-Mk in [14] and our Alg. 3. and our Alg. 3. n ≤ 20 n ≤ 10 n ≤ 10 N np N Rd. Rd. SS-Mk Ours Rd. (%) SS-Mk Ours SS-Mk Ours (%) (%) 2 2 2 2 0.00 2 1 1 0.00 1 1 0.00 3 3 3 3 0.00 22 4 4 0.00 4 4 0.00 5 5 5 5 0.00 3 7 7 7 7 0.00 2 8 8 0.00 32 32 0.00 2 4 9 3 29 29 0.00 2 16 16 0.00 96 80 16.67 2 5 25 5 118 110 6.78 2 256 192 25.00 256 192 25.00 3 6 27 3 197 188 4.57 2 768 512 33.33 896 768 14.29 2 7 49 7 305 269 11.80 2 1792 1152 35.71 2304 1920 16.67 4 8 81 3 1067 998 6.47 2 4608 2816 38.89 4608 3840 16.67 3 9 125 5 1450 1315 9.31 2 12800 11264 12.00 12800 11264 12.00 7 10 128 2 2942 2942 0.00 2 27648 21504 22.22 31744 28672 9.68 3 11 343 7 5072 4728 6.78 2 63488 49152 22.58 63488 57344 9.68 625 54 13489 12140 10.00 12 6 2 163840 122880 25.00 192512 184320 4.26 729 3 22801 20411 10.48 13 2 376832 327680 13.04 385024 368640 4.26 1024 210 48126 48126 0.00 14 2 770048 737280 4.26 770048 737280 4.26 2401 74 63354 62254 1.74 15 5 2 2162688 1900544 12.12 2162688 2162688 0.00 3125 5 108175 97265 10.09 16 12 2 4325376 3801088 12.12 4325376 4325376 0.00 4096 2 278526 278526 0.00 6561 38 377375 330236 12.49 8192 213 655358 655358 0.00 5 16807 7 688713 704693 -2.32 19683 39 of gates. The results are shown in Tables 4 and 5 for 1443791 1259711 12.75 n ≤ 10 and n ≤ 20, respectively, where columns three and four show the numbers of gates for the SS-Mk and TABLE 5 our Alg. 3, and column five shows the reduction of our Comparison of the number of gates without buffers for Alg. 3. For all N’s except for N = 75, our Alg. 3 has sorting N = np inputs for n ≤ 20 via the SS-Mk in [14] no more gates than the SS-Mk in [14]. There are up to and our Alg. 3. 13% and 23% fewer gates than the SS-Mk in [14] for n ≤ 10 and n ≤ 20, respectively. This means our sorting n ≤ 20 network takes better advantage of larger basic sorters. N np SS-Mk Ours Rd. (%) We also remark that using a larger sorter size n may 2 2 2 2 0.00 reduce the number of gates for sorting N = np inputs. 3 3 3 3 0.00 For all common N’s for n ≤ 10 in Table 4 and n ≤ 20 in 5 5 5 5 0.00 7 7 7 7 0.00 Table 5, the same number of gates is needed, since the 11 11 11 11 0.00 same sorter size n is used. For all remaining N’s except 13 13 13 13 0.00 N = 39 17 17 17 17 0.00 for in Table 4, there is a corresponding larger 19 19 19 19 0.00 9 N’s in Table 5 with fewer gates. For N =3 = 19683 in 25 52 118 110 6.78 3 N = 134 = 28561 27 3 197 188 4.57 Table 4 and in Table 5, the latter has 2 about 1% more gates than the former, but accounts for 49 7 305 269 11.80 121 112 1117 917 17.91 45% more inputs. 125 53 1450 1315 9.31 2 169 13 1814 1454 19.85 289 172 3970 3074 22.57 6 CONCLUSION 361 192 5501 4205 23.56 4 625 5 13489 12140 10.00 6 In this work, we proposed a new merging algorithm 729 3 22801 20411 10.48 3 based on n-sorters for parallel sorting networks, where 1331 11 29107 26668 8.38 2197 133 54703 50763 7.20 4 n is prime. Based on the n-way merging, we also 2401 7 63354 62254 1.74 proposed a merge sorting algorithm. Our sorting algo- 3125 55 108175 97265 10.09 4913 173 156812 143443 8.53 rithm is a direct generalization of odd-even merge sort 3 6859 19 239590 221052 7.74 with n-sorters as basic blocks. By using larger sorters 4 14641 11 564513 562214 0.41 (2 ≤ n ≤ 20), the number of sorters as well as the 16807 75 688713 704693 -2.32 latency is reduced greatly. In comparison with other 28561 134 1230724 1271788 -3.34 multiway sorting networks in [14], our implementation has a smaller latency and fewer sorters for wide ranges of N ≤ 1.46 × 104. We also showed an application of sorting networks implemented by linearly scaling sorters number of gates can be greatly reduced by using larger in threshold logic and have a similar conclusion that the sorters. 12 APPENDIX A of all m lists. According to Lemma 3.1, the outputs of PROOFS each list are sorted. Assume any two adjacent wires s (i−1) A.1 Proof of Lemma 3.1 and s +1 in list j are sorted after stage i − 1, xj,s ≤ (i−1) Proof: The proof of the lemma can be reduced to xj,s+1 for 1 ≤ j ≤ n and 1 ≤ s ≤ m − 1. We will show (i) (i) showing that for l > 0 any two wires s,s + l ∈ Zm of that xj,s ≤ xj,s+1 for 1 ≤ j ≤ n and 1 ≤ s ≤ m − 1. each list are sorted as shown in Fig. 4(b). We prove the According to Lemma 3.2, for a prime m, there are three lemma by contradiction. The inputs satisfy xj,s ≤ xj,s+l cases of two adjacent sorters S1 and S2 as shown in for j ∈ Zn and s,s + l ∈ Zm. Suppose there exist k ∈ Zn Fig. 5(a)-(c). Z ′ ′ (i−1) (i−1) and s,s + l ∈ m such that xk,s > xk,s+l. Since the sorter 1) For case I, let yj,1 and yj,2 be the two adjacent for xk,s k ∈ Zm acts as a permutation of the index k, we wires in list j connected by adjacent two sorters in denote such permutation of the sorter connecting wire s stage i − 1 for a ≤ j ≤ b. According to Lemma 3.1 as f : {1, ··· ,n}→{1, ··· ,n}. Because f is bijection, (n =2), the outputs of each list are sorted. −1 a (i−1) −1 an inverse f exists. Then we have xf (t),s+l ≥ 2) For case II, there is an additional single wire yb+1 − ′ ′ ′ (i−1) (i) xf 1(t),s = xt,s ≥ xk,s > xk,s+l for k ≤ t ≤ n, where the connected by S2. If y =1, we have y =1. a b+1,1 b+1,1 “≥ ” is because the inputs are sorted and the “=” is due The last connection of S2 can be removed without to the permutation. There are n−k+1 inputs of xf −1(t),s+l changing the order of others in S2. S1 and the − ′ satisfying xf 1(t),s+l > xk,s+l. However, at most n − k revised S2 reduce to case I and the outputs are ′ ′ outputs satisfy xt,s+l > xk,s+l for t ∈{k+1, k+2, ··· ,n}, (i−1) sorted according to Lemma 3.1. If yb+1,1 = 0, we resulting in a contradiction. Hence, all lists are self sorted (i−1) after applying n-sorters. have yb,1 =0. This is because they are connected by the same sorter in stage i − 1. Then, we have (i) (i) A.2 Proof of Lemma 3.2 ya,1 = ya,2 = 0, which are sorted outputs in list a. (i−1) (i−1) (i) (i) Proof: First, we show that the first connections of Remove yb+1,1,yb,1 ,ya,1, and ya,2, the remaining adjacent two sorters belong to either the same list or of S1 and S2 reduce to a smaller configuration adjacent two lists. Let (j, t1) and (j + l,t2) be the first of case II. With recursively applying the above connections of adjacent two sorters S1 and S2, respec- approach, S1 and S2 either reduce to a smaller tively, where (j, t) denotes wire t in list j. If l > 1, case I or a single wire, both of which gives sorted the connection of S1 in list j + l − 1 should be wire outputs. m; otherwise, S2 would have a valid connection in list 3) For case III, there is an additional single wire (i−1) j + l. For lists j to j + l − 2, only wires m in each list are ya−1,m connected by the first sorter. Similarly, the connected by S1, since wire m can be connected to the two sorters can be reduced to either a case I or a preceding list only by a (m−1)-spaced sorter. Hence, S1 smaller configuration of case III and the outputs of is the last (m − 1)-spaced sorter in stage 1 and S2 does two adjacent wires in each list are sorted. not exist. Similarly, we can show that the last connections Assume all lists are self-sorted after stage i − 1, we have (i−1) (i−1) of adjacent two sorters S1 and S2 belong to either the xj,1 ≤ ··· ≤ xj,m for 1 ≤ j ≤ n. For stage 1 ≤ i ≤ m same list or adjacent two lists. This gives us a total of ⌈ 2 ⌉, all wires in lists j =2, ··· ,n − 1 have connections four cases as shown in Fig. 5, where b ≥ a+1 for Fig. 5(a)- (i) (i) with some sorters. We have xj,k ≤ xj,k for j =2, ··· ,n−1 (c), and b ≥ a for Fig. 5(d) such that S1 and S2 have a and k = 1, ··· ,m − 1. Hence, lists j = 2, ··· ,n − 1 size of at least two. (i−1) (i−1) are self-sorted after stage i. For list 1, x1,i−1 ≤ x2,1 If m is prime, no adjacent two sorters belong to (i−1) (i−1) (i) (i) and x1,i−1 ≤ x1,i , we have x1,i−1 ≤ x1,i. We have case IV, which is equivalent to showing that m is a (i) (i) (i) composite number if case IV in Fig. 5 exists. Assume hx1,1, x1,2, ··· , x1,i−1i, since list 1 is self-sorted after stage (i−1) (i) two adjacent sorters S1 and S2 belong to case IV. Let the i − 1 and x1,k = x1,k for k = 1, ··· ,i − 1. We also S1 (j, m) (i) (i) (i) first connection of be and the last connection have x1,i, x1,i+1, ··· , x1,mi. Hence, list 1 is self-sorted of S2 be (j + p, 1). The last connection of S1 satisfies after stage i, x(i) , x(i) , ··· , x(i) i. Due to symmetry, list (k + 1)p ≡ 0 mod m. We have m | (k + 1)p. Since case 1,1 1,i+1 1,m n is also self-sorted after stage i, x(i) , x(i) , ··· , x(i) i. IV is not possible in the first stage, we have p A.4 Proof of Lemma 3.3 [7] D.-L. Lee and K. E. Batcher, “A multiway merge sorting network,” IEEE Transactions on Parallel and Distributed Systems, vol. 6, no. 2, i−1 Proof: In stage i − 1, there are n sorted lists of pp. 211–215, 1995. n values with respect to each q (q = 1, ··· ,np−i). Since [8] B. Parker and I. Parberry, “Constructing sorting networks from k- sorters,” Information Processing Letters, vol. 33, no. 3, pp. 157–162, the outputs of each merging network are sorted after 1989. stage i − 1, we can replace each merging network by [9] R. Beigel and J. Gill, “Sorting n objects with a k-sorter,” IEEE an ni-sorter. According to Lemma 3.1, the outputs of Transactions on Computers, vol. 39, no. 5, pp. 714–716, 1990. (i) [10] T. Nakatani, S.-T. Huang, B. W. Arden, and S. K. Tripathi, “k-way each new formed list after stage i are sorted, xj,q ≤ bitonic sort,” IEEE Transactions on Computers, vol. 38, no. 2, pp. (i) (i) i 283–288, 1989. x p−i−1 ≤ x p−i−1 for j =1, ··· ,n . Since the j,n +q j,(n−1)n +q [11] D. Lee and K. E. Batcher, “On sorting multiple bitonic sequences,” corresponding wires in the new lists are connected by the in Proc. International Conference on Parallel Processing (ICPP 1994)., i (i) (i) vol. 1, 1994, pp. 121–125. same n -sorter in stage i − 1, we have xj,q ≤ xj+1,q for i (i) (i) i [12] T. Leighton, “Tight bounds on the complexity of parallel sorting,” j =1, ··· ,n −1. Hence, rj,q ≥ rj+1,q for j =1, ··· ,n −1. in Proc. The sixteenth annual ACM symposium on Theory of Comput- For r(i) = n > r(i) ≥ ··· ≥ r(i) > 0 = ing. ACM, 1984, pp. 71–80. s,q s+1,q s+l,q [13] K. J. Liszka and K. E. Batcher, “A generalized bitonic sorting (i) Proc. International Conference on Parallel Processing rs+l+1,q for l ≤ n, it is equivalent to prove that network,” in (i) (i) i (ICPP 1993), vol. 1, 1993, pp. 105–108. xj+n,q =1 if xj,(n−1)np−i−1+q =1 for j ∈{1, ··· ,n − n}. [14] Q. Gao and Z. Liu, “Sloping-and-shaking,” Science in China Series For any q ∈ {1, ··· ,np−1−i} in stage i, there are ni E: Technological Sciences, vol. 40, no. 3, pp. 225–234, 1997. (i) [15] L. Zhao, Z. Liu, and Q. Gao, “An efficient multiway merging lists of n values. Suppose xj,(n−1)np−i−1+q = 0 for algorithm,” Science in China Series E: Technological Sciences, vol. 41, (i) no. 5, pp. 543–551, 1998. j ≤ s and x p−i−1 = 1. If t (t ≤ s) zeros of s+1,(n−1)n +q [16] R. Drysdale, III and F. H. Young, “Improved divide sort merge (i) sorting networks,” SIAM Journal on Computing, vol. 4, no. 3, pp. xj,(n−1)np−i−1+q are from the same list of the original n (i) 264–270, 1975. sorted lists, there are at most t+1 zeros of xj,q from that [17] D. C. Van Voorhis, “An economical construction for sorting (i) networks,” in Proceedings of the May 6-10, 1974, national computer same list. Since xj,(n−1)np−i−1+q = 0 for j ≤ s are from conference and exposition. ACM, 1974, pp. 921–927. at most n original lists, there are at most s + n zeros [18] S. Muroga, Threshold Logic and Its Applications. New York: WILEY- in x(i) , implying that x(i) = 1. Hence, x(i) = 1 if INTERSCIENCE, 1971. j,q s+n,q j+n,q [19] V. Beiu, J. Peperstraete, and R. Lauwereins, “Enhanced thresh- (i) i − − Proceedings of the third xj,(n−1)np i 1+q =1 for j ∈{1, ··· ,n − n}. old gate fan-in reduction algorithms,” in international conference on Young computer scientists. Tsinghua University Press, 1993, pp. 339–342. A.5 Proof of Theorem 3.2 Proof: In stage 1, all outputs with respect to the operation of the same Alg. 1 are sorted. For any q ∈ {1, ··· ,np−1−i} in stage i, according to Lemma 3.3, at most n consecutive lists are not full of zeros. All preceding lists are all-zero lists and all following lists are all-one lists. Hence, the combining network in stage i is to sort n lists of n values, which is reduced to Alg. 1. In stage p − 1, we have q =1 and the single sorted list, (i) (i) (i) (i) (i) hx1,q, x1,np−i−1+q, ··· , x1,(n−1)np−i−1+q, x2,q, x2,np−i−1+q, (i) (i) (i) ··· , x2,(n−1)np−i−1+q, ··· , xni,q, xni,np−i−1+q, ··· , (i) p xni,(n−1)np−i−1+qi, contains n values, implying all inputs are sorted as a single list. REFERENCES [1] D. E. Knuth, “The Art of Computer Programming. Sorting and Searching, vol. III,” 1973. [2] K. E. Batcher, “Sorting networks and their applications,” in Proc. The Spring Joint Computer Conference. ACM, 1968, pp. 307–314. [3] K. J. Liszka and K. E. Batcher, “A modulo merge sorting network,” in Proc. Fourth Symposium on the Frontiers of Massively Parallel Computation, 1992. IEEE, 1992, pp. 164–169. [4] K. E. Batcher, “On bitonic sorting networks,” in Proc. Internation Conference on Parallel Processing (ICPP), 1990, pp. 376–379. [5] M. Ajtai, J. Koml´os, and E. Szemer´edi, “An o(n log n) sorting network,” in Proc. The fifteenth annual ACM symposium on Theory of Computing. ACM, 1983, pp. 1–9. [6] A. Farmahini-Farahani, H. J. Duwe, M. J. Schulte, and K. Comp- ton, “Modular design of high-throughput, low-latency sorting units,” IEEE Transactions on Computers, vol. 62, no. 7, pp. 1389– 1402, 2013.