Slide 1 Slide 2 Areas done in textbook: Algorithms and Applications • Sorting Algorithms • Numerical Algorithms • Image Processing • Searching and Optimization ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 3 Slide 4 Chapter 10 Potential Speedup O(nlogn) optimal for any sequential sorting algorithm without using special properties of the numbers. Sorting Algorithms Best we can expect based upon a sequential sorting algorithm using n processors is - rearranging a list of numbers into increasing (strictly non- decreasing) order. O(nlogn ) Optimal parallel time complexity = ------------------------- = O(logn ) n Has been obtained but the constant hidden in the order notation extremely large. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 5 Slide 6 Compare-and-Exchange Sorting Algorithms Message-Passing Compare and Exchange Compare and Exchange Version 1 Form the basis of several, if not most, classical sequential sorting P1 sends A to P2, which compares A and B and sends back B to P1 algorithms. if A is larger than B (otherwise it sends back A to P1): Sequence of steps Two numbers, say A and B, are compared. If A > B, A and B are P P 1 1 2 exchanged, i.e.: A Send(A) B If A > B send(B) else send(A) If A > B load A if (A > B) { 2 else load B temp = A; Compare 3 A = B; B = temp; } ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 7 Slide 8 Alternative Message Passing Method Note on Precision of Duplicated Computations Version 2 For P1 to send A to P2 and P2 to send B to P1. Then both processes Previous code assumes that the if condition, A > B, will return the perform compare operations. P1 keeps the larger of A and B and P2 same Boolean answer in both processors. keeps the smaller of A and B: Different processors operating at different precision could P P 1 1 2 conceivably produce different answers if real numbers are being A Send(A) B compared. Send(B) 2 If A > B load A This situation applies to anywhere computations are duplicated in Compare Compare 3 If A > B load B different processors to reduce message passing, or to make the 3 code SPMD. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 9 Slide 10 Data Partitioning Merging Two Sublists — Version 2 (Version 1) p processors and n numbers. n/p numbers assigned to each P1 P2 Original processor: Merge numbers Merge Original 98 98 numbers 98 98 Keep 88 80 80 88 higher 80 43 43 80 numbers P1 P2 42 42 (final 50 50 numbers) Keep 43 43 Merge lower 88 Original 88 42 50 numbers 50 42 88 88 numbers 28 28 98 Keep (final 28 28 Original 50 50 88 numbers) 25 25 25 25 numbers 28 28 higher 80 numbers 25 25 50 43 43 98 Return 42 80 42 Final Original 28 lower numbers 28 numbers 43 numbers 25 42 25 ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 11 Slide 12 Original sequence: 4 2 7 8 5 1 3 6 4 2 7 8 5 1 3 6 Bubble Sort 2 4 7 8 5 1 3 6 2 4 7 8 5 1 3 6 Phase 1 First, largest number moved to the end of list by a series of Place 2 4 7 8 5 1 3 6 largest number compares and exchanges, starting at the opposite end. 2 4 7 5 8 1 3 6 2 4 7 5 1 8 3 6 Actions repeated with subsequent numbers, stopping just before 2 4 7 5 1 3 8 6 the previously positioned number. 2 4 7 5 1 3 6 8 2 4 7 5 1 3 6 8 Phase 2 Place 2 4 7 5 1 3 6 8 In this way, the larger numbers move (“bubble”) toward one end, next largest 2 4 5 7 1 3 6 8 number Time ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 13 Slide 14 Parallel Bubble Sort Iteration could start before previous iteration finished if does not overtake previous bubbling action: Phase 1 Time Complexity 1 n – 1 n(n – 1) 1 Number of compare and exchange operations = i = å -------------------- Phase 2 i = 1 2 2 1 Indicates a time complexity of O(n2) given that a single compare- Time 2 1 and-exchange operation has a constant complexity, O(1). Phase 3 3 2 1 3 2 1 Phase 4 4 3 2 1 ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 15 Slide 16 Odd-Even Transposition Sort Odd-Even (Transposition) Sort Sorting eight numbers Variation of bubble sort. P0 P1 P2 P3 P4 P5 P6 P7 Step Operates in two alternating phases, even phase and odd phase. 0 4 2 7 8 5 1 3 6 1 2 4 7 8 1 5 3 6 Even phase 2 2 4 7 1 8 3 5 6 3 2 4 1 7 3 8 5 6 Even-numbered processes exchange numbers with their right neighbor. Time 4 2 1 4 3 7 5 8 6 5 1 2 3 4 5 7 6 8 Odd phase 6 1 2 3 4 5 6 7 8 Odd-numbered processes exchange numbers with their right 7 1 2 3 4 5 6 7 8 neighbor. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 17 Slide 18 Parallelizing Mergesort Using tree allocation of processes Mergesort Unsorted list 4 2 7 8 5 1 3 6 P0 A classical sequential sorting algorithm using divide-and-conquer 4 2 7 8 5 1 3 6 P P approach. Unsorted list first divided into half. Each half is again Divide 0 4 list divided into two. Continued until individual numbers are obtained. 4 2 7 8 5 1 3 6 P0 P2 P4 P6 4 2 7 8 5 1 3 6 P P P P P P P P Then pairs of numbers combined (merged) into sorted list of two 0 1 2 3 4 5 6 7 numbers. Pairs of these lists of four numbers are merged into sorted 2 4 7 8 1 5 3 6 P0 P2 P4 P6 lists of eight numbers. This is continued until the one fully sorted list Merge 2 4 7 8 1 3 5 6 P0 P4 is obtained. 1 2 3 4 5 6 7 8 P0 Sorted list Process allocation ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. ã Barry Wilkinson 2003. This material is for sole and exclusive use of students enrolled at Western Carolina University. It is not to be sold, reproduced, or generally distributed. Slide 19 Slide 20 Quicksort Analysis Very popular sequential sorting algorithm that performs well with an average sequential time complexity of O(nlogn). Sequential First list divided into two sublists. All the numbers in one sublist Sequential time complexity is O(nlogn). arranged to be smaller than all the numbers in the other sublist. Achieved by first selecting one number, called a pivot, against which Parallel every other number is compared.