Algorithmic Analysis and Sorting, Part Two Friday Four Square! 4:15PM, Outside Gates An Initial Idea: Selection Sort An Initial Idea: Selection Sort 4 1 2 7 6 An Initial Idea: Selection Sort 4 1 2 7 6 An Initial Idea: Selection Sort 4 1 2 7 6 An Initial Idea: Selection Sort 1 4 2 7 6 An Initial Idea: Selection Sort 1 4 2 7 6 An Initial Idea: Selection Sort 1 4 2 7 6 An Initial Idea: Selection Sort 1 4 2 7 6 An Initial Idea: Selection Sort 1 4 2 7 6 An Initial Idea: Selection Sort 1 4 2 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 7 6 An Initial Idea: Selection Sort 1 2 4 6 7 An Initial Idea: Selection Sort 1 2 4 6 7 Notes on Selection Sort ● Selection sort is O(n2) in the worst case. ● Selection sort is O(n2) in the best case. ● Notation: Selection sort is Θ(n2). Another Idea: Insertion Sort Another Idea: Insertion Sort 7 4 2 1 6 Another Idea: Insertion Sort 7 4 2 1 6 Another Idea: Insertion Sort 7 4 2 1 6 Another Idea: Insertion Sort 7 4 2 1 6 Another Idea: Insertion Sort 4 7 2 1 6 Another Idea: Insertion Sort 4 7 2 1 6 Another Idea: Insertion Sort 4 7 2 1 6 Another Idea: Insertion Sort 4 7 2 1 6 Another Idea: Insertion Sort 4 2 7 1 6 Another Idea: Insertion Sort 4 2 7 1 6 Another Idea: Insertion Sort 2 4 7 1 6 Another Idea: Insertion Sort 2 4 7 1 6 Another Idea: Insertion Sort 2 4 7 1 6 Another Idea: Insertion Sort 2 4 7 1 6 Another Idea: Insertion Sort 2 4 1 7 6 Another Idea: Insertion Sort 2 4 1 7 6 Another Idea: Insertion Sort 2 1 4 7 6 Another Idea: Insertion Sort 2 1 4 7 6 Another Idea: Insertion Sort 1 2 4 7 6 Another Idea: Insertion Sort 1 2 4 7 6 Another Idea: Insertion Sort 1 2 4 7 6 Another Idea: Insertion Sort 1 2 4 7 6 Another Idea: Insertion Sort 1 2 4 6 7 Another Idea: Insertion Sort 1 2 4 6 7 Insertion Sort in Code void InsertionSort(Vector<int>& v) { for (int i = 0; i < v.size(); i++) { for (int j = i – 1; j >= 0; j--) { if (v[j] < v[j + 1]) break; swap(v[j], v[j + 1]); } } } How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 Work done: O(n) How Fast is Insertion Sort? 7 6 4 2 1 How Fast is Insertion Sort? 7 6 4 2 1 How Fast is Insertion Sort? 7 6 4 2 1 How Fast is Insertion Sort? 7 6 4 2 1 How Fast is Insertion Sort? 6 7 4 2 1 How Fast is Insertion Sort? 6 7 4 2 1 How Fast is Insertion Sort? 6 7 4 2 1 How Fast is Insertion Sort? 6 7 4 2 1 How Fast is Insertion Sort? 6 4 7 2 1 How Fast is Insertion Sort? 6 4 7 2 1 How Fast is Insertion Sort? 4 6 7 2 1 How Fast is Insertion Sort? 4 6 7 2 1 How Fast is Insertion Sort? 4 6 7 2 1 How Fast is Insertion Sort? 4 6 7 2 1 How Fast is Insertion Sort? 4 6 2 7 1 How Fast is Insertion Sort? 4 6 2 7 1 How Fast is Insertion Sort? 4 2 6 7 1 How Fast is Insertion Sort? 4 2 6 7 1 How Fast is Insertion Sort? 2 4 6 7 1 How Fast is Insertion Sort? 2 4 6 7 1 How Fast is Insertion Sort? 2 4 6 7 1 How Fast is Insertion Sort? 2 4 6 7 1 How Fast is Insertion Sort? 2 4 6 1 7 How Fast is Insertion Sort? 2 4 6 1 7 How Fast is Insertion Sort? 2 4 1 6 7 How Fast is Insertion Sort? 2 4 1 6 7 How Fast is Insertion Sort? 2 1 4 6 7 How Fast is Insertion Sort? 2 1 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 How Fast is Insertion Sort? 1 2 4 6 7 Work done: O(n2) Notes on Insertion Sort ● Insertion sort is O(n) in the best case. ● Insertion sort is O(n2) in the worst case. ● On average, insertion sort is roughly twice as fast as selection sort. ● In a random array, every element is about halfway away from where it belongs. ● So for the kth element, about (k – 1) / 2 other elements must be looked at. Selection Sort vs Insertion Sort Size Selection Sort Insertion Sort 10000 0.304 0.160 20000 1.218 0.630 30000 2.790 1.427 40000 4.646 2.520 50000 7.395 4.181 60000 10.584 5.635 70000 14.149 8.143 80000 18.674 10.333 90000 23.165 12.832 Thinking About O(n2) 14 6 3 9 7 16 2 15 5 10 8 11 1 13 12 4 T(n) Thinking About O(n2) 14 6 3 9 7 16 2 15 5 10 8 11 1 13 12 4 T(n) 14 6 3 9 7 16 2 15 5 10 8 11 1 13 12 4 Thinking About O(n2) 14 6 3 9 7 16 2 15 5 10 8 11 1 13 12 4 T(n) 14 6 3 9 7 16 2 15 5 10 8 11 1 13 12 4 T(½n) ≈ ¼T(n) T(½n) ≈ ¼T(n) Thinking About O(n2) 14 6 3 9 7 16 2 15 5 10 8 11 1 13 12 4 T(n) 2 3 6 7 9 14 15 16 1 4 5 8 10 11 12 13 T(½n) ≈ ¼T(n) T(½n) ≈ ¼T(n) Thinking About O(n2) 14 6 3 9 7 16 2 15 5 10 8 11 1 13 12 4 T(n) 2 3 6 7 9 14 15 16 1 4 5 8 10 11 12 13 T(½n) ≈ ¼T(n) T(½n) ≈ ¼T(n) ItIt takestakes roughlyroughly ½T(n)½T(n) toto sortsort eacheach halfhalf separately!separately! The Key Insight: Merge The Key Insight: Merge 2 4 7 8 10 1 3 5 6 9 The Key Insight: Merge 2 4 7 8 10 1 3 5 6 9 The Key Insight: Merge 2 4 7 8 10 1 3 5 6 9 The Key Insight: Merge 2 4 7 8 10 3 5 6 9 1 The Key Insight: Merge 2 4 7 8 10 3 5 6 9 1 The Key Insight: Merge 4 7 8 10 3 5 6 9 1 2 The Key Insight: Merge 4 7 8 10 3 5 6 9 1 2 The Key Insight: Merge 4 7 8 10 5 6 9 1 2 3 The Key Insight: Merge 4 7 8 10 5 6 9 1 2 3 The Key Insight: Merge 7 8 10 5 6 9 1 2 3 4 The Key Insight: Merge 7 8 10 5 6 9 1 2 3 4 The Key Insight: Merge 7 8 10 6 9 1 2 3 4 5 The Key Insight: Merge 7 8 10 6 9 1 2 3 4 5 The Key Insight: Merge 7 8 10 9 1 2 3 4 5 6 The Key Insight: Merge 7 8 10 9 1 2 3 4 5 6 The Key Insight: Merge 8 10 9 1 2 3 4 5 6 7 The Key Insight: Merge 8 10 9 1 2 3 4 5 6 7 The Key Insight: Merge 10 9 1 2 3 4 5 6 7 8 The Key Insight: Merge 10 9 1 2 3 4 5 6 7 8 The Key Insight: Merge 10 1 2 3 4 5 6 7 8 9 The Key Insight: Merge 1 2 3 4 5 6 7 8 9 10 “Split Sort” “Split Sort” void SplitSort(Vector<int>& v) { Vector<int> left, right; for (int i = 0; i < v.size() / 2; i++) left += v[i]; for (int j = v.size() / 2; j < v.size(); j++) right += v[i]; InsertionSort(left); InsertionSort(right); Merge(left, right, v); } Performance Comparison Size Selection Sort Insertion Sort “Split Sort” 10000 0.304 0.160 0.161 20000 1.218 0.630 0.387 30000 2.790 1.427 0.726 40000 4.646 2.520 1.285 50000 7.395 4.181 2.719 60000 10.584 5.635 2.897 70000 14.149 8.143 3.939 80000 18.674 10.333 5.079 90000 23.165 12.832 6.375 Performance Comparison Size Selection Sort Insertion Sort “Split Sort” 10000 0.304 0.160 0.161 20000 1.218 0.630 0.387 30000 2.790 1.427 0.726 40000 4.646 2.520 1.285 50000 7.395 4.181 2.719 60000 10.584 5.635 2.897 70000 14.149 8.143 3.939 80000 18.674 10.333 5.079 90000 23.165 12.832 6.375 A Better Idea ● Splitting the input in half and merging halves the work.