Bucket Sort: Sorting Integers Sorting (Part II) • The goal: sort N numbers, all between 1 to k. CSE 373 • Example: sort 8 numbers 3,6,7,4,11,3,5,7. All between 1 to 12. Data Structures • The method: Use an array of k queues. Queue j Unit 17 (for 1 ≤ j ≤ k) keeps the input numbers whose value is j. Reading: Section 3.2.6 Radix sort • Each queue is denoted ‘a bucket’. Section 7.6 Mergesort, Section 7.7, Quicksort, Sections 7.8 Lower bound • Scan the list and put the elements in the buckets. • Output the content of the buckets from 1 to k. 2 Bucket Sort: Sorting Integers Radix Sort: Sorting integers • Example: sort 8 numbers 3,6,7,4,11,3,9,7 all • Historically goes back to the 1890 census. between 1 to 12. • Radix sort = multi-pass bucket sort of integers • Step 1: scan the list and put the elements in in the range 0 to BP-1 the queues • Bucket-sort from least significant to most 1 2 3 4 5 6 7 8 9 10 11 12 significant “digit” (base B) 3 4 6 7 9 11 3 7 • Requires P(B+N) operations where P is the • Step 2: concatenate the queues number of passes (the number of base B digits 3 4 6 7 9 11 in the largest possible input number). 3 7 3,3,4,6,7,7,9,11 • If P and B are constants then O(N) time to sort! • Time complexity: O(n+k). 3 4 Radix Sort Example Radix Sort Example After 1st pass Bucket sort Input data st nd Bucket sort After 1 pass by 10’s After 2 pass 478 721 by 1’s digit digit 3 537 3 721 3 9 9 0 1 2 3 4 5 6 7 8 9 123 0 1 2 3 4 5 6 7 8 9 721 537 123 721 721 3 537 478 9 03 721 537 67 478 123 3 123 67 38 67 537 38 478 67 09 123 38 537 123 38 478 38 67 9 38 67 9 478 This example uses B=10 and base 10 digits for simplicity of demonstration. Larger bucket counts should be used in an actual implementation. 5 6 Radix Sort Example Properties of Radix Sort After 2nd pass Bucket sort After 3rd pass by 100’s • Not in-place 3 digit 3 9 9 › needs lots of auxiliary storage. 721 0 1 2 3 4 5 6 7 8 9 38 123 003 123 478 537 721 67 • Stable 537 009 123 › equal keys always end up in same bucket in the 38 038 478 67 067 537 same order. 478 721 • Fast › Time to sort N numbers in the range 0 to BP-1 is Invariant: after k passes the low order k digits are sorted. O(P(B+N)) (P iterations, B buckets in each) 7 8 “Divide and Conquer” Mergesort • Very important strategy in computer science: › Divide problem into smaller parts 8 2 9 4 5 3 1 6 › Independently solve the parts › Combine these solutions to get overall solution • Idea 1: Divide array into two halves, • Divide it in two at the midpoint recursively sort left and right halves, then merge two halves Mergesort • Conquer each side in turn (by • Idea 2 : Partition array into items that are recursively sorting) “small” and items that are “large”, then • Merge two halves together recursively sort the two sets Quicksort 9 10 Mergesort Example Auxiliary Array 8 2 9 4 5 3 1 6 • The merging requires an auxiliary array. Divide 8 2 9 4 5 3 1 6 Divide 2 4 8 9 1 3 5 6 8 2 9 4 5 3 1 6 Divide 1 element 8 2 9 4 5 3 1 6 Merge 2 8 4 9 3 5 1 6 Auxiliary array Merge 2 4 8 9 1 3 5 6 Merge 1 2 3 4 5 6 8 9 11 12 Auxiliary Array Auxiliary Array • The merging requires an auxiliary array. • The merging requires an auxiliary array. 2 4 8 9 1 3 5 6 2 4 8 9 1 3 5 6 1 Auxiliary array 1 2 3 4 5 Auxiliary array 13 14 Merging Merging first i j normal second i j Right completed first target target Left completed copy i j first target 15 16 Merging Recursive Mergesort Merge(A[], T[] : integer array, left, right : integer) : { Mergesort(A[], T[] : integer array, left, right : integer) : { mid, i, j, k, l, target : integer; if left < right then mid := (right + left)/2; mid := (left + right)/2; i := left; j := mid + 1; target := left; Mergesort(A,T,left,mid); while i < mid and j < right do Mergesort(A,T,mid+1,right); if A[i] < A[j] then T[target] := A[i] ; i:= i + 1; Merge(A,T,left,right); else T[target] := A[j]; j := j + 1; } target := target + 1; MainMergesort(A[1..n]: integer array, n : integer) : { if i > mid then //left completed// T[1..n]: integer array; for k := left to target-1 do A[k] := T[k]; Mergesort[A,T,1,n]; if j > right then //right completed// } k : = mid; l := right; while k > i do A[l] := A[k]; k := k-1; l := l-1; for k := left to target-1 do A[k] := T[k]; } 17 18 Iterative Mergesort Iterative Mergesort Merge by 1 Merge by 1 Merge by 2 Merge by 2 Merge by 4 Merge by 4 Merge by 8 Merge by 8 Merge by 16 Need of a last copy 19 20 Iterative Mergesort Mergesort Analysis IterativeMergesort(A[1..n]: integer array, n : integer) : { • Let T(N) be the running time for an //precondition: n is a power of 2// i, m, parity : integer; array of N elements T[1..n]: integer array; m := 2; parity := 0; • Mergesort divides array in half and calls while m < n do for i = 1 to n – m + 1 by m do itself on the two halves. After returning, if parity = 0 then Merge(A,T,i,i+m-1); else Merge(T,A,i,i+m-1); it merges both halves using a temporary parity := 1 – parity; array m := 2*m; if parity = 1 then for i = 1 to n do A[i] := T[i]; • Each recursive call takes T(N/2) and } merging takes O(N) How do you handle non-powers of 2? How can the final copy be avoided? 21 22 Mergesort Recurrence Properties of Mergesort Relation • The recurrence relation for T(N) is: • Not in-place › T(1) < a › Requires an auxiliary array (O(n) extra • base case: 1 element array constant time space) › T(N) < 2T(N/2) + bN • Stable • Sorting N elements takes › Make sure that left is sent to target on – the time to sort the left half – plus the time to sort the right half equal values. – plus an O(N) time to merge the two halves • Iterative Mergesort reduces copying. • T(N) = O(n log n) (see Lecture 5 Slide17) 23 24 Quicksort “Four easy steps” • Quicksort uses a divide and conquer strategy, • To sort an array S but does not require the O(N) extra space 1. If the number of elements in S is 0 or 1, that MergeSort does then return. The array is sorted. › Partition array into left and right sub-arrays 2. Pick an element v in S. This is the pivot • Choose an element of the array, called pivot • the elements in left sub-array are all less than pivot value. • elements in right sub-array are all greater than pivot 3. Partition S-{v} into two disjoint subsets, S1 › Recursively sort left and right sub-arrays ≤ ≥ = {all values x v}, and S2 = {all values x v}. › Concatenate left and right sub-arrays in O(1) time 4. Return QuickSort(S1), v, QuickSort(S2) 25 26 The steps of QuickSort Details, details select pivot value S 81 43 31 57 13 75 • Implementing the actual partitioning 92 0 65 26 • Picking the pivot partition S S1 0 S2 › want a value that will cause |S1| and |S2| to 43 31 75 13 65 81 26 57 92 be non-zero, and close to equal in size if possible QuickSort(S1) and S1 S2 QuickSort(S2) 0 13 26 31 43 57 65 75 81 92 • Dealing with cases where an element equals the pivot S 0 13 26 31 43 57 65 75 81 92 Voila! S is sorted [Weiss] 27 28 Quicksort Partitioning Partitioning:Choosing the pivot • Need to partition the array into left and right sub- • One implementation (there are others) arrays › median3 finds pivot and sorts left, center, › the elements in left sub-array are ≤ pivot right › elements in right sub-array are ≥ pivot • Median3 takes the median of leftmost, middle, and • How do the elements get to the correct partition? rightmost elements • An alternative is to choose the pivot randomly (need a › Choose an element from the array as the pivot random number generator; “expensive”) › Make one pass through the rest of the array and • Another alternative is to choose the first element (but can be very bad. Why?) swap as needed to put elements in partitions › Swap pivot with next to last element 29 30 Partitioning in-place Example Choose the pivot as the median of three › Set pointers i and j to start and end of array 0 1 2 3 4 5 6 7 8 9 8 1 4 9 0 3 5 2 7 6 › Increment i until you hit element A[i] > pivot Median of 0, 6, 8 is 6.