Introduction sorting Chapter 7 fundamental task in data management well-studied problem in computer science Sorting basic problem given an of items where each item contains a key, rearrange the items so that the keys appear in ascending order the key may be only of the item begin sorted e.g., the item could be an entire block of information about a student, while the search key might be only the student's name 2 Introduction Sorting Algorithms we will assume sorts the array to sort contains only integers insertion sort defined < and > operators (comparison-based sorting) selection sort all array positions contain data to be sorted bubble sort why ? the entire sort can be performed in _________________ Shellsort: ___________________ number of elements is relatively small: < a few million sorts given a list of n items, we will use the heapsort notation to indicate that the search for a mergesort does not follow that of quicksort sorts that cannot be performed in main memory a lower bound on sorting by _ comparison may be performed on disk or tape sorting algorithms: count sort known as sorting string sorts 3 4 Sorting Sorting given keys to sort, there are possible of cost model for sorting the keys! the basic operations we will count are and e.g, given and the keys , there are ___________ possible permutations: if there are array accesses that are not associated with comparisons or swaps, we need to count them, too programming notes if the objects being sorted are large, we should swap brute force enumeration of permutations is not ______________ to the objects, rather than the objects computationally once themselves polymorphism: general-purpose sorting algorithms vs templated algorithms 5 6 Insertion Sort Insertion Sort insertion sort algorithm simple passes for (p = 1; p < N; p++) { in pass , , we move to its correct tmp = location among for (j = p; (j > 0) && (tmp < ); j--) { for passes to , insertion sort ensures that the swap and elements in positions to are in sorted order } at the end of pass , the elements in positions to are in = tmp sorted order } 7 8 Insertion Sort Insertion Sort to avoid the full swap, we can use the following code: example for (p = 1; p < N; p++) { tmp = for (j = p; (j > 0) && (tmp < ); j--) { = } = tmp } 9 10 Insertion Sort Insertion Sort example analysis position 0 1 2 3 4 5 best case: the keys are already initial sequence 42 6 1 54 0 7 comparisons, no swaps p = 1 6 42 1 54 0 7 worst case: the keys are in sorted order p = 2 6 1 42 54 0 7 expected (average) case: ? 1 6 42 54 0 7 p = 3 1 6 42 54 0 7 p = 4 1 6 42 0 54 7 1 6 0 42 54 7 1 0 6 42 54 7 0 1 6 42 54 7 p = 5 0 1 6 42 7 54 0 1 6 7 42 54 11 12 Insertion Sort Insertion Sort analysis number of inversions given that we wish to sort in ascending order, an a list can have between and inversions, the _______________ is any pair that is out of order relative to latter of which occurs when one another: for which but thus, counting inversions also says the worst-case behavior the list of insertion is ________________ what do inversions tell us about the behavior? contains the following inversions: let be the probability space of all permutations of distinct elements with equal _____________________ Theorem: The expected number of inversions in a list swapping an adjacent pair of elements that are out of order taken from is ______________ exactly one inversion thus, the expected complexity of insertion sort is thus, any sort that operates by swapping adjacent terms requires as many swaps as there are inversions quadratic 13 14 Insertion Sort Insertion Sort proof proof (cont.) observe that any pair in a list that is an inversion is in the since there are distinct pairs of lists and their correct order in the of that list reverses, there is a total of list inversions reverse lists inversions 1, 2, 3, 4 0 4, 3, 2, 1 6 inversions among the possible lists of distinct objects 2, 1, 3, 4 1 4, 3, 1, 2 5 3, 2, 1, 4 3 4, 1, 2, 3 3 this means that the expected number of inversions in any given list is this means that if we look at a list and its reverse and count the total number of inversions, then the combined number of inversions is 15 16 Insertion Sort Selection Sort in summary selection sort for randomly ordered arrays of length with distinct keys, find the smallest item and exchange it with the first entry insertion sort uses find the next smallest item and exchange it with the second comparisons and swaps on ___________ entry comparisons and swaps in the _________ find the next smallest item and exchange it with the third case entry comparisons and swaps in the case 17 18 Selection Sort Selection Sort algorithm example for (p = 0; p < N; p++) { position 0 1 2 3 4 m = p initial sequence 42 6 9 54 0 for (j = p+1; j < 0; j++) { after p = 0 0 42 9 54 6 if ( < ) { after p = 1 0 6 9 54 42 m = j } after p = 2 0 6 9 54 42 } after p = 3 0 6 9 42 54 swap and } 19 20 Selection Sort Selection Sort complexity proof comparisons and swaps to sort an array of there is one swap per iteration of the loop, length which is executed times the amount of work is of the input there is one comparison made in each iteration of the selection sort is no faster on input than on _______________ loop, which is executed random input selection sort involves a smaller number of than any of the other sorting algorithms we will consider 21 22 Bubble Sort Bubble Sort bubble sort algorithm read the items from left to right if two items are out of order, swap them not_done = true repeat until sorted while (not_done) { a sweep with swaps means we are done not_done = false for (i = 0 to N - 2) { if ( > ) { swap and not_done = true } } } 23 24 Bubble Sort Bubble Sort example complexity if the data are already sorted, we make only sweep initial sequence 42 6 9 54 0 through the list while-loop 6 42 9 54 0 otherwise, the complexity depends on the number of times 6 9 42 54 0 we execute the while-loop 6 9 42 0 54 since bubble sort swaps items, it will have _______________ worst-case and expected-case while-loop 6 9 0 42 54 complexity while-loop 6 0 9 42 54 while-loop 0 6 9 42 54 25 26 Shellsort Shellsort Shellsort Shellsort Donald L. Shell (1959), A high-speed sorting procedure, suppose we use the increment sequence 15, 7, 5, 3, 1, and Communications of the ACM 2 (7): 3032. have finished the 15-sort and 7-sort swapping only adjacent items dooms us to quadratic worst- then we know that case behavior, so swap items! Shellsort starts with an sequence it uses insertion sort to sort we also know that every -th term starting at , then then every -th term starting at , then then etc. putting these together, we see that every term starting at , after which the array is sorted 27 28 Shellsort Shellsort Shellsort example after we have performed the sort using increment , the array is -sorted: all elements that are terms apart are in the order: the key to Shellsort's is the following fact: an -sorted array remains -sorted after sorting with increment 29 30 Shellsort Shellsort complexity complexity (cont.) a good increment sequence has the the sequences property that for any element , when it is time for the - (T. H. Hibbard, 1963) sort, there are only a few elements to the left of that are ___________ than (V. R. Pratt, 1971) Shell's original increment sequence has ____________ yield worst-case complexity behavior: other sequences yield worst-case complexity , 31 32 Heapsort Heapsort priority queues can be used to sort in can avoid extra array by storing element in original array strategy heap leaves an as each smallest element build binary of elements time is deleted store element in newly opened space, which is no longer perform deleteMin operations elements (smallest first) stored in second array, then used by the heap ____________ back into original array time results in list of order in array requires extra , which doubles memory to achieve increasing order, change ordering of heap to requirement ________ heap when complete, array contains elements in ascending order 33 34 Heapsort Heapsort max heap after buildHeap phase max heap after first deleteMax 35 36 Heapsort Mergesort analysis mergesort is a algorithm building binary heap of elements < comparisons in Vol. III of The Art of Computer Programming, Knuth attributes the algorithm to John von Neumann (1945) total deleteMax operations if heapsort in worst case the idea of mergesort is simple: average case extremely complex to compute divide the array in two sort each half improved to or simply the two subarrays using mergesort heapsort useful if we want to sort the or merging simple since sorted ____________ elements and mergesort can be implemented recursively and non- recursively runs in , worst case 37 38 Mergesort Mergesort merging algorithm takes two arrays, and , and _________ example array also uses three counters: , , initialized to beginning of respective arrays smaller of and copied to when either input array exhausted, the remainder of the other list is to 39 40 Mergesort Mergesort example (cont.) example (cont.) another way to visualize 41 42 Mergesort Mergesort time to merge two lists is at most analysis comparisons running time represented by relation every comparison adds an element to assume is a power of 2 so that list is always divided mergesort easy to characterize ____________ if , only one element to sort for , time to mergesort is _________________ otherwise, mergesort first half and otherwise, time to mergesort numbers is time to perform second half two recursive mergesort of size , plus the time to merge these two halves merge, which is linear problem is divided into smaller problems and solved recursively, and conquered by patching the solutions together 43 44 Mergesort Mergesort analysis (cont.) solving mergesort recurrence relation using ______________ standard recurrence relation can be solved in at least two ways telescoping divide the recurrence through by substitution 45 46 Mergesort Mergesort solving mergesort recurrence relation using telescoping solving mergesort recurrence relation using ______________ (cont.) (cont.) 47 48 Mergesort Mergesort proof by (cont.) proof by induction (cont.) Prove: is equivalent to I.S.: Show: let Base: (or ) by I.H.