Sorting Networks Indranil Banerjee Parallel Sorting: Hardware Level Sorting Networks Parallelism Indranil Banerjee George Mason University [email protected] March 3, 2016 Sorting Networks Indranil Banerjee GMU March 3, 2016 1 / 19 Sorting Networks Hardware Level Parallelism Indranil Banerjee Parallel Sorting: Hardware Level There are mainly two approaches to sorting in parallel: Parallelism 1 Non-oblivious: Comparisons are data dependent Example: Parallel Quicksort, Parallel Merge Sort etc. 2 Oblivious: Comparisons are precomputed and does not depend on the results of previous comparisons. Example: Sorting Networks Sorting Networks Indranil Banerjee GMU March 3, 2016 2 / 19 Sorting Networks Oblivious Sorting Indranil Banerjee Parallel Sorting: Hardware Level Parallelism 1 Since the comparisons are not data dependent, we can precoumpute the comparisons and directly implemented them inside a hardware 2 An oblivious sorting algorithm proceeds in stages 3 Each stage consists of a number of comparisons which occur concurrently 4 We will look at one such algorithm: Batcher's Odd-Even Merge Sort Sorting Networks Indranil Banerjee GMU March 3, 2016 3 / 19 Sorting Networks Odd-Even Merge Sort Indranil Banerjee Parallel Sorting: Hardware Level The Algorithm: X Parallelism OddEvenMergeSort( ) Input: Array X = fx0; x1; :::; xn−1g (Assume n is power of 2) Output: Sorted sequence X 1 XL = fx0; :::; xn=2−1g and XR = fxn=2; :::; xn−1g 2 If n > 1: OddEvenMergeSort(XL) OddEvenMergeSort(XR ) OddEvenMerge(XL; XR ) Recursive Sorting Networks Indranil Banerjee GMU March 3, 2016 4 / 19 Sorting Networks Odd-Even Merge Indranil Banerjee Parallel Sorting: The Algorithm: X Hardware OddEvenMerge( ) Level Parallelism Input: An arry X whose two halvs XL and XR are sorted (Assume n = jXLj = jXR j is power of 2) Output: Sorted sequence X 1 If n > 2 Then: Let XEven = fx0; x2; :::; xng and XOdd = fx1; x3; :::xn−1g i OddEvenMerge(XEven) ii OddEvenMerge(XOdd ) iii Pardo: Compare(x2i−1; x2i ) While (1 ≤ i ≤ (n − 2)=2) 2 Compare(x0; x1) Sorting Networks Indranil Banerjee GMU March 3, 2016 5 / 19 Sorting Networks Sorting Network Indranil Banerjee Parallel A Comparator: Sorting: Hardware Level Parallelism Source: http://parallelcomp.uw.hu/ch09lev1sec2.html Sorting Networks Indranil Banerjee GMU March 3, 2016 6 / 19 Sorting Networks Sorting Network Indranil Banerjee Series Parallel Comparisons: Parallel Sorting: Hardware Level Parallelism Figure: What is this sorting algorithm? Source: http://www.cs.cmu.edu/ tcortina/15110m14/ps9/ Sorting Networks Indranil~ Banerjee GMU March 3, 2016 7 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network: Parallel Sorting: Hardware Level Parallelism M2 M4 M2 M8 M2 M4 M2 Figure: The comparator blocks are individual merging networks Sorting Networks Indranil Banerjee GMU March 3, 2016 8 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merging Network: Parallel Sorting: Hardware Level M2 Parallelism M4 Figure: Merging networks for n = 2; 4 Sorting Networks Indranil Banerjee GMU March 3, 2016 9 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 3 Hardware Level Parallelism 6 7 1 2 9 5 8 Sorting Networks Indranil Banerjee GMU March 3, 2016 10 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 3 Hardware Level Parallelism 6 1 7 2 9 5 8 Sorting Networks Indranil Banerjee GMU March 3, 2016 11 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 6 3 7 2 8 5 9 Sorting Networks Indranil Banerjee GMU March 3, 2016 12 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 3 6 7 2 5 8 9 Sorting Networks Indranil Banerjee GMU March 3, 2016 13 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 3 6 7 2 5 8 9 Sorting Networks Indranil Banerjee GMU March 3, 2016 14 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 3 2 5 6 7 8 9 Sorting Networks Indranil Banerjee GMU March 3, 2016 15 / 19 Sorting Networks Sorting Network Indranil Banerjee Batcher's Odd-Even Merge Sort Network (Expanded): Parallel Sorting: 1 Hardware Level Parallelism 2 3 5 6 7 8 9 Sorting Networks Indranil Banerjee GMU March 3, 2016 16 / 19 Sorting Networks Proving Correctness: The 0-1 Principle Indranil Banerjee Parallel Sorting: Hardware Level Parallelism 1 The correctness of the any oblivious sorting algorithm can be proven using the 0-1-principle 2 0-1-principle: If a sorting network sorts every sequence of 0's and 1's, then it sorts every arbitrary sequence of values. Complexity? Sorting Networks Indranil Banerjee GMU March 3, 2016 17 / 19 Sorting Networks Proving Correctness: The 0-1 Principle Indranil Banerjee Parallel Sorting: Hardware Level 1 The correctness of the any oblivious sorting algorithm can Parallelism be proven using the 0-1-principle 2 0-1-principle: If a sorting network sorts every sequence of 0's and 1's, then it sorts every arbitrary sequence of values. Complexity? Can be answered directly by looking at the network. 2 1 Size: O(n log n) (This is the serial runtime) 2 2 Depth: O(log n) (This is the parallel runtime) Sorting Networks Indranil Banerjee GMU March 3, 2016 18 / 19 Sorting Networks Q&A Indranil Banerjee Parallel Sorting: Hardware Level Parallelism 1 Can we have sorting networks with O(log n) depth and O(n log n) size? 2 How do we implement such networks? Sorting Networks Indranil Banerjee GMU March 3, 2016 19 / 19.