CS202 Fall 2012 Lecture 11/27 Sorting 1. Bogo Sort

Home , Bogosort, Bubble sort, Insertion sort, Merge sort, Quicksort, Selection sort

11/27/12

Lecture outline

bogo sort

bubble sort

selection sort CS202 Fall 2012 Lecture 11/27 insertion sort

merge sort Sorting quick sort

Prof. Tanya Berger-Wolf the quick select problem

1 2

Sorting

Sorting = putting objects in order in array/list one of the fundamental problems in computer science comparison-based sorting: must determine order through comparison operations on the input data: <, >, compareTo, … 1. Bogo sort

15 2 8 1 17 10 12 5 0 1 2 3 4 5 6 7

1 11/27/12

Bogo sort Bogo sort code bogo sort: orders a list of values by repetitively shuffling them and public static void bogoSort(int[] a) { while (!isSorted(a)) { checking if they are sorted shuffle(a); } more specifically: }  scan the list, seeing if it is sorted // Returns true if array a's elements  if not, shuffle the values in the list and repeat // are in sorted order. public static boolean isSorted(int[] a) { for (int i = 0; i < a.length - 1; i++) { This sorting algorithm has terrible performance! if (a[i] > a[i+1]) {

 Can we deduce its runtime? return false; } } return true; }

5 6

Bogo sort code, helpers Bogo sort runtime

// Shuffles an array of ints by randomly swapping each // element with an element ahead of it in the array. How long should we expect bogo sort to take? public static void shuffle(int[] a) {  related to probability of shuffling into sorted order for (int i = 0; i < a.length - 1; i++) { // pick random number in [i+1, a.length-1]  assuming shuffling code is fair, probability of having sorted equals inclusive int range = a.length-1 - (i + 1) + 1; 1 / (number of permutations of n elements) int j = (int)(Math.random() * range + (i + 1)); swap(a, i, j); } } // Swaps a[i] with a[j]. private static void swap(int[] a, int i, int j) { if (i == j) return; int temp = a[i];  bogo sort takes roughly factorial time to run a[i] = a[j]; – note that if array is initially sorted, bogo finishes quickly! a[j] = temp; }  it should be clear that this is not satisfactory...

7 8

2 11/27/12

Bubble sort

bubble sort: orders a list of values by repetitively comparing neighboring elements and swapping their positions if necessary

more specifically:

 scan the list, exchanging adjacent elements if they are not in 2. Bubble sort relative order; this bubbles the highest value to the top  scan the list again, bubbling up the second highest value

 repeat until all elements have been placed in their proper order

9 10

"Bubbling" largest element "Bubbling" largest element

Traverse a collection of elements Traverse a collection of elements

 Move from the front to the end  Move from the front to the end

 "Bubble" the largest value to the end using pair-wise comparisons  "Bubble" the largest value to the end using pair-wise comparisons and swapping and swapping

1 2 3 4 5 6 1 2 3 4 5 6 Swap 7742 4277 35 12 101 5 42 7735 Swap 3577 12 101 5

11 12

3 11/27/12

"Bubbling" largest element "Bubbling" largest element

Traverse a collection of elements Traverse a collection of elements

 Move from the front to the end  Move from the front to the end

 "Bubble" the largest value to the end using pair-wise comparisons  "Bubble" the largest value to the end using pair-wise comparisons and swapping and swapping

1 2 3 4 5 6 1 2 3 4 5 6 Swap 42 35 7712 1277 101 5 42 35 12 77 101 5

No need to swap

13 14

"Bubbling" largest element "Bubbling" largest element

Traverse a collection of elements Traverse a collection of elements

 Move from the front to the end  Move from the front to the end

 "Bubble" the largest value to the end using pair-wise comparisons  "Bubble" the largest value to the end using pair-wise comparisons and swapping and swapping

1 2 3 4 5 6 1 2 3 4 5 6

42 35 12 77 1015 Swap 101 5 42 35 12 77 5 101

Largest value correctly placed

15 16

4 11/27/12

Bubble sort code Bubble sort runtime public static void bubbleSort(int[] a) { Running time (# comparisons) for input size n: for (int i = 0; i < a.length; i++) { for (int j = 1; j < a.length - i; j++) { // swap adjacent out-of-order elements if (a[j-1] > a[j]) { swap(a, j-1, j); } } } }

 number of actual swaps performed depends on the data; out-of- order data performs many swaps

17 18

Selection sort

selection sort: orders a list of values by repetitively putting a particular value into its final position

more specifically:

 find the smallest value in the list 3. Selection sort  switch it with the value in the first position  find the next smallest value in the list

 switch it with the value in the second position

 repeat until all values are in their proper places

19 20

5 11/27/12

Selection sort example Selection sort example 2

Index 0 1 2 3 4 5 6 7

Value 27 63 1 72 64 58 14 9

1st pass 1 63 27 72 64 58 14 9

2nd pass 1 9 27 72 64 58 14 63

3rd pass 1 9 14 72 64 58 27 63

…

21 22

Selection sort code Selection sort runtime public static void selectionSort(int[] a) { Running time for input size n: for (int i = 0; i < a.length; i++) { // find index of smallest element  in practice, a bit faster than bubble sort. Why? int min = i; for (int j = i + 1; j < a.length; j++) { if (a[j] < a[min]) { min = j; } } // swap smallest element with a[i] swap(a, i, min); } }

23 24

6 11/27/12

Insertion sort

insertion sort: orders a list of values by repetitively inserting a particular value into a sorted subset of the list

more specifically:

 consider the first item to be a sorted sublist of length 1 4. Insertion sort  insert the second item into the sorted sublist, shifting the first item if needed

 insert the third item into the sorted sublist, shifting the other items as needed

 repeat until all values have been inserted into their proper positions

25 26

Insertion sort Insertion sort example

Simple sorting algorithm.

 n-1 passes over the array

 At the end of pass i, the elements that occupied A[0]…A[i] originally are still in those spots and in sorted order.

2 15 8 1 17 10 12 5

0 1 2 3 4 5 6 7 after 2 8 15 1 17 10 12 5 pass 2 0 1 2 3 4 5 6 7

1 2 8 15 17 10 12 5 after 0 1 2 3 4 5 6 7 pass 3 27 28

7 11/27/12

Insertion sort example 2 Insertion sort example 2

Names: Fred, Alice, David, Bill, and Carol

29 30

Insertion sort example 2 Worst-case for insertion sort

31 32

8 11/27/12

Insertion sort code Insertion sort runtime public static void insertionSort(int[] a) { worst case: reverse-ordered elements in array. for (int i = 1; i < a.length; i++) { int temp = a[i];

// slide elements down to make room for a[i] int j = i; while (j > 0 && a[j - 1] > temp) { best case: array is in sorted ascending order. a[j] = a[j - 1]; j--; }

a[j] = temp; } average case: each element is about halfway in order. }

33 34

Average case analysis Comparing sorts

We've seen "simple" sorting algorithms so far, such as: Given an array A of elements, an inversion is an ordered pair (i, j) such

 selection sort that i < j, but

 insertion sort A[i] > A[j]. (out of order elements) Assume no duplicate elements. comparisons swaps Theorem: The average number of inversions in an array of n distinct selection n2/2 n elements is n (n - 1) / 4. Corollary: Any algorithm that sorts by exchanging adjacent elements worst: n2/2 worst: n2/2 requires O(n2) time on average. insertion best: n best: 0

They all use nested loops and perform approximately n2 comparisons They are relatively inefficient

35 36

9 11/27/12

CS202 Fall 2012 5. Merge sort Lecture

Sorting

Prof. Tanya Berger-Wolf

38 http://xkcd.com/584/ 37

Merge sort Merge sort merge sort: orders a list of values by recursively dividing the list in half until each sub-list has one element, then recombining Merge sort idea:  Invented by John von Neumann in 1945  Divide the array into two halves.

 Recursively sort the two halves (using merge sort).

 Use merge to combine the two arrays. more specifically:

 divide the list into two roughly equal parts mergeSort(0, n/2-1) mergeSort(n/2, n-1)  recursively divide each part in half, continuing until a part contains only one element

 merge the two parts into one sorted list sort sort  continue to merge parts as the recursion unfolds merge(0, n/2, n-1) This is a "divide and conquer" algorithm.

39 40

10 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

41 42

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14

98 23

43 44

11 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14

98 23 98 23

23 Merge Merge

45 46

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14

98 23 98 23 45 14

23 98 23 98 Merge

47 48

12 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14

23 98 23 98 14

Merge Merge

49 50

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14

23 98 14 45 23 98 14 45

Merge

51 52

13 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14

23 98 14 45 23 98 14 45

14 14 23

Merge Merge

53 54

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14

23 98 14 45 23 98 14 45

14 23 45 14 23 45 98

Merge Merge

55 56

14 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 98 23 45 14 6 67

23 98 14 45 23 98 14 45

14 23 45 98 14 23 45 98

57 58

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 6 67 98 23 45 14 6 67

23 98 14 45 23 98 14 45 6

14 23 45 98 Merge 14 23 45 98 Merge

59 60

15 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

98 23 45 14 6 67 98 23 45 14 6 67 33 42

23 98 14 45 6 67 23 98 14 45 6 67

14 23 45 98 Merge 14 23 45 98

61 62

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 23 98 14 45 6 67 33

14 23 45 98 Merge 14 23 45 98 Merge

63 64

16 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 Merge 14 23 45 98

Merge

65 66

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 6 14 23 45 98 6 33

Merge Merge

67 68

17 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 14 23 45 98 6 33 42 67

Merge Merge

69 70

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67 14 23 45 98 6 33 42 67

Merge Merge

71 72

18 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67 14 23 45 98 6 33 42 67

6 14 6 14 23

Merge Merge

73 74

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67 14 23 45 98 6 33 42 67

6 14 23 33 6 14 23 33 42

Merge Merge

75 76

19 11/27/12

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67 14 23 45 98 6 33 42 67

6 14 23 33 42 45 6 14 23 33 42 45 67

Merge Merge

77 78

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42 23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67 14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98

Merge

79 80

20 11/27/12

Merge sort example 2 98 23 45 14 6 67 33 42

13 6 21 18 9 4 8 20

0 7

13 6 21 18 9 4 8 20

0 3 4 7

13 6 21 18 9 4 8 20

0 1 2 3 4 5 6 7

13 6 21 18 9 4 8 20

0 1 2 3 4 5 6 7

6 13 18 21 4 9 8 20

0 1 2 3 4 5 6 7 6 14 23 33 42 45 67 98 6 13 18 21 4 8 9 20 0 3 4 7

4 6 8 9 13 18 20 21

0 7

81 82

Merging two sorted arrays Merge sort code merge operation: public static void mergeSort(int[] a) {  Given two sorted arrays, merge operation produces a sorted array with all int[] temp = new int[a.length]; mergeSort(a, temp, 0, a.length - 1); the elements of the two arrays } A 6 13 18 21 B 4 8 9 20 private static void mergeSort(int[] a, int[] temp, int left, int right) { if (left >= right) { // base case C 4 6 8 9 13 18 20 21 return; } // sort the two halves int mid = (left + right) / 2; Running time of merge: O(n), where n is the number of elements in mergeSort(a, temp, left, mid); the merged array. mergeSort(a, temp, mid + 1, right); when merging two sorted parts of the same array, we'll need a temporary array // merge the sorted halves into a sorted whole to store the merged whole merge(a, temp, left, right); }

21 11/27/12

Merge code Merge sort runtime private static void merge(int[] a, int[] temp, int left, int right) { Merge sort int mid = (left + right) / 2; int count = right - left + 1; T(n) = 2T(n/2) + n int l = left; // counter indexes for L, R log n int r = mid + 1; a = 2, b = 2, f(n) = n. 2 // main loop to copy the halves into the temp array i i for (int i = 0; i < count; i++) 2 ⋅ (n/2 ) = n log2n if (r > right) { // finished right; ∑ use left i=0 temp[i] = a[l++]; } else if (l > mid) { // finished left; use right temp[i] = a[r++]; } else if (a[l] < a[r]) { // left is smaller (better) temp[i] = a[l++]; } else { // right is smaller General form: T(n) €= aT(n/b ) + f(n€) (better) temp[i] = a[r++]; } € logbn // copy sorted temp array back into main array i n for (int i = 0; i < count; i++) { a f( ) a[left + i] = temp[i]; ∑ i } b } i=0

85 € €

Quick sort

quick sort: orders a list of values by partitioning the list around one element called a pivot, then sorting each partition

 invented by British computer scientist C.A.R. Hoare in 1960

more specifically: 6. Quick sort  choose one element in the list to be the pivot (= partition element)  organize the elements so that all elements less than the pivot are to its left and all greater are to its right

 apply the quick sort algorithm (recursively) to both partitions

87 88

22 11/27/12

Quick sort pseudo-code Quick sort, continued

For correctness, it's okay to choose any pivot. Let S be the input set.

For efficiency, one of following is best case, the other worst case: 1.If |S| = 0 or |S| = 1, then return.

 pivot partitions the list roughly in half

 pivot is greatest or least element in list 2.Pick an element v in S. Call v the pivot.

Which case above is best? 3.Partition S – {v} into two disjoint groups:

• S1 = {x ∈ S – {v} | x ≤ v}

We will divide the work into two methods: • S2 = {x ∈ S – {v} | x ≥ v}

 quickSort – performs the recursive algorithm

 partition – rearranges the elements into two partitions 4.Return { quicksort(S1), v, quicksort(S2) }

89 90

Quick sort illustrated How to choose a pivot

first element pick a pivot  bad if input is sorted or in reverse sorted order

 bad if input is nearly sorted 40 18 37 2  variation: particular element (e.g. middle element) 32 6 35 10 12 17 random element

 even a malicious agent cannot arrange a bad input partition 6 17 40 37 median of three elements 10 12 2 18 32 35  choose the median of the left, right, and center elements quicksort quicksort 2 6 10 12 17 18 32 35 37 40 combine 2 6 10 12 17 18 32 35 37 40

91 92

23 11/27/12

Partitioning algorithm Quick sort code

public static void quickSort(int[] a) { The basic idea: quickSort(a, 0, a.length - 1); 1.Move the pivot to the rightmost position. } private static void quickSort(int[] a, int min, int max) { 2.Starting from the left, find an element ≥ pivot. Call the position i. if (min >= max) { // base case; no need to sort return; 3.Starting from the right, find an element ≤ pivot. Call the position j. } 4.Swap S[i] and S[j]. // choose pivot -- we'll use the first element (might be bad!) int pivot = a[min]; swap(a, min, max); // move pivot to end // partition the two sides of the array int middle = partition(a, min, max - 1, pivot); // restore the pivot to its proper location swap(a, middle, max); // recursively sort the left and right partitions quickSort(a, min, middle - 1); quickSort(a, middle + 1, max); }

8 1 4 9 0 3 5 2 7 6

0 9

93 94

Quick sort code, cont'd.

// partitions a with elements < pivot on left and // elements > pivot on right; // returns index of element that should be swapped with pivot private static int partition(int[] a, int min, int max, int pivot) { while (true) { // move markers i,j inward to find misplaced elements CS202 Fall 2012 while (a[min] < pivot) { min++; } while (a[max] > pivot) { max--; } Lecture if (min >= max) break; Sorting swap(a, min, max); min++; max--; Prof. Tanya Berger-Wolf } return min; }

95 96

24 11/27/12

"Median of three" pivot Special cases

What happens when the array contains many duplicate elements?

9 17 3 12 8 7 21 1 What happens when the array is already sorted (or nearly sorted) to 0 pick pivot 7 begin with?

Small arrays 1 17 3 12 8 7 21 9  Quicksort is slower than insertion sort when is N is small (say, N ≤ 0 7 i j 20).

 Optimization: Make |A| ≤ 20 the base case and use insertion sort 1 7 3 12 8 17 21 9 algorithm on arrays of that size or smaller. 0 i j 7

1 7 3 8 12 17 21 9 0 j i swap S[i] 7 with S[right] 1 7 3 8 9 17 21 12 0 7 97 98

Quick sort runtime Quick sort runtime, cont'd.

Worst case: pivot is the smallest (or largest) element all the time. Assume each of the sizes for S1 are equally likely. 0 ≤ |S1| ≤ N-1. T(n) = T(n-1) + cn T(n-1) = T(n-2) + c(n-1) N ⎛ N−1 ⎞ 2 1 T(N) = T(1) + c i = O(N ) ⎜ ⎟ T(n-2) = T(n-3) + c(n-2) ∑ T(N) = ⎜ ∑[T(i) + T(N -i -1)]⎟ + cN … i=2 ⎝ N i=0 ⎠ T(2) = T(1) + 2c ⎛ 2 N−1 ⎞ ⎜ ⎟ = ⎜ ∑T(i)⎟ + cN € ⎝ N i=0 ⎠ ⎛ N−1 ⎞ Best case: pivot is the median N T(N) = ⎜2 T(i)⎟ + cN2 T(n) = 2 T(n/2) + cn ⎜ ∑ ⎟ ⎝ i=0 ⎠ T(n) = O(n log n) N−2 (N −1) T(N −1) = 2∑T(i) + c(N -1)2 i=0

€

99 100

25 11/27/12

Quick sort runtime, cont'd. Quick sort runtime summary

N T(N) = (N + 1) T(N −1) + 2cN O(n log n) on average. O(n2) worst case. T(N) T(N −1) 2c = + divide N + 1 N N + 1 equation T(N -1) T(N − 2) 2c € = + by N(N+1) comparisons N N -1 N € T(N - 2) T(N − 3) 2c = + merge O(n log n) N −1 N - 2 N −1 … average: O(n log n) € quick T(2) T(1) 2c 2 = + worst: O(n ) 3 2 3 € N+1 T(N) T(1) 1 = + 2c∑ N + 1 2 i=3 i € T(N) = O(N log N) ≈loge (N+1) – 3/2

101 102 € €

Sorting practice problem Sorting practice problem

Consider the following array of int values. Consider the following array: [ 7, 17, 22, -1, 9, 6, 11, 35, -3]

[22, 11, 34, -5, 3, 40, 9, 16, 6] Each of the following is a view of a sort-in-progress on the elements. Which sort is which?  (If the algorithm is a multiple-loop algorithm, the array is shown (f) Write the contents of the array after all the partitioning of quick after a few of these loops have completed. If the algorithm is sort has finished (before any recursive calls). recursive, the array is shown after the recursive calls have finished on each sub-part of the array.)  Assume that the quick sort algorithm chooses the first element as Assume that the median of three elements (first, middle, and last) its pivot at each pass. is chosen as the pivot. (a) [-3, -1, 6, 17, 9, 22, 11, 35, 7] (b) [-1, 7, 17, 22, -3, 6, 9, 11, 35] (c) [ 9, 22, 17, -1, -3, 7, 6, 35, 11] (d) [-1, 7, 6, 9, 11, -3, 17, 22, 35] (e) [-3, 6, -1, 7, 9, 17, 11, 35, 22] (f) [-1, 7, 17, 22, 9, 6, 11, 35, -3]

103 104

26 11/27/12

Sorting practice problem Selection problem; quickSelect

For the following questions, indicate which of the six sorting algorithms Recall: the Selection problem will successfully sort the elements in the least amount of time.  Find the kth smallest element in an array a  The algorithm chosen should be the one that completes fastest, without crashing.  Assume that the quick sort algorithm chooses the first element as its pivot at each pass. quickSelect(a, k):  Assume stack overflow occurs on 5000+ stacked method calls. 1.If a.length = 1, then k=1 and return the element.

 (a) array size 2000, random order 2.Pick a pivot v ∈ a.  (b) array size 500000, ascending order 3.Partition a – {v} into a1 (left side) and a2 (right side).  (c) array size 100000, descending order – special constraint: no extra memory may be allocated! (O(1) • if k ≤ a1.length, then the kth smallest element must be in a1. So

storage) return quickSelect(a1, k).  (d) array size 1000000, random order • else if k = 1 + a1.length, return the pivot v.  (e) array size 10000, ascending order – special constraint: no extra memory may be allocated! (O(1) • Otherwise, the kth smallest element is in a2. Return quickSelect(a2,

storage) k - a1.length - 1).

105 106

Quick select runtime

average case:

…

= O(N)

worst case: same as Quick sort, O(N2)

107