Unit 7 Sorting Algorithms Simple Sorting algorithms Quicksort Improving Quicksort Overview of Sorting Algorithms Given a collection of items we want to arrange them in an increasing or decreasing order. You probably have seen a number of sorting algorithms including ¾ selection sort ¾ insertion sort ¾ bubble sort ¾ quicksort ¾ tree sort using BST's In terms of efficiency: ¾ average complexity of the first three is O(n2) ¾ average complexity of quicksort and tree sort is O(n lg n) ¾ but its worst case is still O(n2) which is not acceptable In this section, we ¾ review insertion, selection and bubble sort ¾ discuss quicksort and its average/worst case analysis ¾ show how to eliminate tail recursion ¾ present another sorting algorithm called heapsort Unit 7- Sorting Algorithms 2 Selection Sort Assume that data ¾ are integers ¾ are stored in an array, from 0 to size-1 ¾ sorting is in ascending order Algorithm for i=0 to size-1 do x = location with smallest value in locations i to size-1 swap data[i] and data[x] end Complexity If array has n items, i-th step will perform n-i operations First step performs n operations second step does n-1 operations ... last step performs 1 operatio. Total cost : n + (n-1) +(n-2) + ... + 2 + 1 = n*(n+1)/2 . Algorithm is O(n2). Unit 7- Sorting Algorithms 3 Insertion Sort Algorithm for i = 0 to size-1 do temp = data[i] x = first location from 0 to i with a value greater or equal to temp shift all values from x to i-1 one location forwards data[x] = temp end Complexity Interesting operations: comparison and shift i-th step performs i comparison and shift operations Total cost : 1 + 2 + ... + (n-1) + n = n*(n+1)/2. Algorithm is O(n2). Unit 7- Sorting Algorithms 4 Bubble Sort n passes each pass swaps out of order adjacent elements Algorithm for end = size-1 to 0 do for i = 0 to end-1 do if data[i] > data[i+1] swap data[i] and data[i+1] end end Complexity Each step in inside for-loop performs end # of operations. Therefore, the total cost of the algorithm is n + (n-1) + ... + 1 = n*(n+1)/2. Algorithm is O(n2). Unit 7- Sorting Algorithms 5 Tree Sort Insert each element in a BST or AVL tree. Traverse the tree inorder and place the elements back into the array. Algorithm tree = an empty BST or AVL tree for i = 0 to size-1 do insert data[i] in tree end for i = 0 to size-1 do get the next inorder item in tree store the item in data[i] end Complexity Inserting n items in the bst or AVL tree: O(n log(n)), on the average. Traversing the tree and inserting back the items in the array : O(n) time. Average cost : O(n log(n) + n) = O(n log(n)). If BST's are used, worst case cost: O(n2). Problem: algorithm needs an additional O(n) space for the tree. Unit 7- Sorting Algorithms 6 Quicksort Quicksort is a 'Divide and Conquer' (recursive method) Algorithm – pick one element of the array as the pivot – partition the array into two regions: • the left region that has the items less than the pivot • the right region that has the items greater or equal to pivot – apply quicksort to the left region – apply quicksort to the right region Unit 7- Sorting Algorithms 7 Quicksort Code typedef long int Item; // It returns the final position of the pivot void quicksort( Item a[], int first, int last ){ int partition( Item a[], int first, int last ) int pos; // position of pivot { Item pivot = a[first]; // pivot if (first < last) int left = first, right = last; { // array has more than 1 item pos = partition( a, first, last ); while (left < right) { quicksort(a, first, pos-1); // search from right for item <= pivot quicksort(a, pos+1, last); while ( a[right] > pivot ) } right--; } //search from left for item >= pivot while ( left < right && a[left] <= pivot ) left++; // swap items if left and right have not cross if ( left < right ) { Item tmp =a[left]; a[left] = a[right]; a[right] = tmp; } } // place pivot in correct position a[first] = a[right]; a[right] = pivot; return right; } Unit 7- Sorting Algorithms 8 Quicksort – Partition Algorithm The partition algorithm do the following: ¾ picks the first item as the pivot item ¾ divides the array into regions – the left region that contains all items that are <= pivot – the right region that contains all items that are > pivot ¾ places the pivot at the right slot (no need to move it any more) ¾ returns the pivot’s final position – after that, quicksort has to recursively sort the left region (the part before the pivot) and the right region (part after the pivot) Partition starts searching from the two ends of the array and swaps the items that are in the wrong region. ¾ When the two searches meet, the array is partitioned ¾ Then the pivot is placed in the right position Unit 7- Sorting Algorithms 9 Quicksort - Example initial: 17 32 68 16 14 15 44 22 1st partition 17 32 68 16 14 15 44 22 lr 17 15 68 16 14 32 44 22 lr 17 15 14 16 68 32 44 22 rl 16 15 14 17 68 32 44 22 2nd partition 16 15 14 l r 14 15 16 3rd partition 14 15 rl 4th partition 68 32 44 22 I r 22 32 44 68 5th partition 22 32 44 rl 22 32 44 6th partition 32 44 rl Unit 7- Sorting Algorithms 32 44 10 Quicksort – Time Complexity Average Time ¾ Partition is O(n): it's just a loop through the array ¾ Assuming that we split the array in half each time, we need: – 2 recursive calls of half the array, – 4 of 1/4 of the array, and so on. ¾ So the algorithm has at the most log(n) levels. ¾ Therefore, the average time complexity of the algorithm is O(n log n) Average Space ¾ It can be shown that the max number of frames on the stack is log n ¾ So average space is O(log n) Worst Time ¾ Array is sorted. Partition splits the array into an empty region and a region with n-1 items ¾ Then time = n + (n-1) + … + 2 + 1 which is O(n2) Worst Space ¾ It will create n stack frames. So space is O(n) Unit 7- Sorting Algorithms 11 Improvements of Quicksort How can we improve the worst time case? ¾ select as pivot the median of the left, right and middle items ¾ significantly reduces the probability of the worst case How can we improve the worst space case? ¾ Remove tail recursion (recursive call done as the last operation) and replace it with iteration With these improvements, time/space complexity fof Quicksort is: Average Worst (for time) time O(nlgn) O(n2) space O(lg n) O(1) Further improvements: ¾ Quicksort is inefficient on small arrays ¾ Stop using Quicksort when partition size is small (i.e. < 50) use insertion sort for this part of the array) Unit 7- Sorting Algorithms 12.