Sorting Networks
Indranil Banerjee
Parallel Sorting: Hardware Level Sorting Networks Parallelism
Indranil Banerjee
George Mason University [email protected]
March 3, 2016
Sorting Networks Indranil Banerjee GMU March 3, 2016 1 / 19 Sorting Networks Hardware Level Parallelism Indranil Banerjee
Parallel Sorting: Hardware Level There are mainly two approaches to sorting in parallel: Parallelism
1 Non-oblivious: Comparisons are data dependent
Example: Parallel Quicksort, Parallel Merge Sort etc.
2 Oblivious: Comparisons are precomputed and does not depend on the results of previous comparisons. Example: Sorting Networks
Sorting Networks Indranil Banerjee GMU March 3, 2016 2 / 19 Sorting Networks Oblivious Sorting Indranil Banerjee
Parallel Sorting: Hardware Level Parallelism 1 Since the comparisons are not data dependent, we can precoumpute the comparisons and directly implemented them inside a hardware 2 An oblivious sorting algorithm proceeds in stages 3 Each stage consists of a number of comparisons which occur concurrently 4 We will look at one such algorithm: Batcher’s Odd-Even Merge Sort
Sorting Networks Indranil Banerjee GMU March 3, 2016 3 / 19 Sorting Networks Odd-Even Merge Sort Indranil Banerjee
Parallel Sorting: Hardware Level The Algorithm: X Parallelism OddEvenMergeSort( )
Input: Array X = {x0, x1, ..., xn−1} (Assume n is power of 2) Output: Sorted sequence X
1 XL = {x0, ..., xn/2−1} and XR = {xn/2, ..., xn−1} 2 If n > 1: OddEvenMergeSort(XL) OddEvenMergeSort(XR ) OddEvenMerge(XL, XR ) ← Recursive
Sorting Networks Indranil Banerjee GMU March 3, 2016 4 / 19 Sorting Networks Odd-Even Merge Indranil Banerjee
Parallel Sorting: The Algorithm: X Hardware OddEvenMerge( ) Level Parallelism Input: An arry X whose two halvs XL and XR are sorted (Assume n = |XL| = |XR | is power of 2) Output: Sorted sequence X 1 If n > 2 Then: Let XEven = {x0, x2, ..., xn} and XOdd = {x1, x3, ...xn−1} i OddEvenMerge(XEven) ii OddEvenMerge(XOdd ) iii Pardo: Compare(x2i−1, x2i ) While (1 ≤ i ≤ (n − 2)/2)
2 Compare(x0, x1)
Sorting Networks Indranil Banerjee GMU March 3, 2016 5 / 19 Sorting Networks Sorting Network Indranil Banerjee
Parallel A Comparator: Sorting: Hardware Level Parallelism
Source: http://parallelcomp.uw.hu/ch09lev1sec2.html
Sorting Networks Indranil Banerjee GMU March 3, 2016 6 / 19 Sorting Networks Sorting Network Indranil Banerjee Series Parallel Comparisons:
Parallel Sorting: Hardware Level Parallelism
Figure: What is this sorting algorithm?
Source: http://www.cs.cmu.edu/ tcortina/15110m14/ps9/ Sorting Networks Indranil~ Banerjee GMU March 3, 2016 7 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network: Parallel Sorting: Hardware Level Parallelism M2
M4
M2
M8 M2
M4
M2
Figure: The comparator blocks are individual merging networks
Sorting Networks Indranil Banerjee GMU March 3, 2016 8 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merging Network: Parallel Sorting: Hardware Level M2 Parallelism
M4
Figure: Merging networks for n = 2, 4
Sorting Networks Indranil Banerjee GMU March 3, 2016 9 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network (Expanded):
Parallel Sorting: 3 Hardware Level Parallelism 6
7
1
2
9
5
8
Sorting Networks Indranil Banerjee GMU March 3, 2016 10 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 3 Hardware Level Parallelism 6
1
7
2
9
5
8
Sorting Networks Indranil Banerjee GMU March 3, 2016 11 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 6
3
7
2
8
5
9
Sorting Networks Indranil Banerjee GMU March 3, 2016 12 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 3
6
7
2
5
8
9
Sorting Networks Indranil Banerjee GMU March 3, 2016 13 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 3
6
7
2
5
8
9
Sorting Networks Indranil Banerjee GMU March 3, 2016 14 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 3
2
5
6
7
8
9
Sorting Networks Indranil Banerjee GMU March 3, 2016 15 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher’s Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 2
3
5
6
7
8
9
Sorting Networks Indranil Banerjee GMU March 3, 2016 16 / 19 Sorting Networks Proving Correctness: The 0-1 Principle Indranil Banerjee
Parallel Sorting: Hardware Level Parallelism 1 The correctness of the any oblivious sorting algorithm can be proven using the 0-1-principle 2 0-1-principle: If a sorting network sorts every sequence of 0’s and 1’s, then it sorts every arbitrary sequence of values.
Complexity?
Sorting Networks Indranil Banerjee GMU March 3, 2016 17 / 19 Sorting Networks Proving Correctness: The 0-1 Principle Indranil Banerjee
Parallel Sorting: Hardware Level 1 The correctness of the any oblivious sorting algorithm can Parallelism be proven using the 0-1-principle 2 0-1-principle: If a sorting network sorts every sequence of 0’s and 1’s, then it sorts every arbitrary sequence of values.
Complexity? Can be answered directly by looking at the network. 2 1 Size: O(n log n) (This is the serial runtime) 2 2 Depth: O(log n) (This is the parallel runtime)
Sorting Networks Indranil Banerjee GMU March 3, 2016 18 / 19 Sorting Networks Q&A Indranil Banerjee
Parallel Sorting: Hardware Level Parallelism
1 Can we have sorting networks with O(log n) depth and O(n log n) size? 2 How do we implement such networks?
Sorting Networks Indranil Banerjee GMU March 3, 2016 19 / 19