CS 758/858: Algorithms ■ COVID Prof. Wheeler Ruml Algorithms TA Sumanta Kashyapi This Class Complexity http://www.cs.unh.edu/~ruml/cs758 4 handouts: course info, schedule, slides, asst 1 2 online handouts: programming tips, formulas 1 physical sign-up sheet/laptop (for grades, piazza) Wheeler Ruml (UNH) Class 1, CS 758 – 1 / 25 COVID ■ COVID Algorithms This Class Complexity ■ check your Wildcat Pass before coming to campus ■ if you have concerns, let me know Wheeler Ruml (UNH) Class 1, CS 758 – 2 / 25 ■ COVID Algorithms ■ Algorithms Today ■ Definition ■ Why? ■ The Word ■ The Founder This Class Complexity Algorithms Wheeler Ruml (UNH) Class 1, CS 758 – 3 / 25 Algorithms Today ■ ■ COVID web: search, caching, crypto Algorithms ■ networking: routing, synchronization, failover ■ Algorithms Today ■ machine learning: data mining, recommendation, prediction ■ Definition ■ Why? ■ bioinformatics: alignment, matching, clustering ■ The Word ■ ■ The Founder hardware: design, simulation, verification ■ This Class business: allocation, planning, scheduling Complexity ■ AI: robotics, games Wheeler Ruml (UNH) Class 1, CS 758 – 4 / 25 Definition ■ COVID Algorithm Algorithms ■ precisely defined ■ Algorithms Today ■ Definition ■ mechanical steps ■ Why? ■ ■ The Word terminates ■ The Founder ■ input and related output This Class Complexity What might we want to know about it? Wheeler Ruml (UNH) Class 1, CS 758 – 5 / 25 Why? ■ ■ COVID Computer scientist 6= programmer Algorithms ◆ ■ Algorithms Today understand program behavior ■ Definition ◆ have confidence in results, performance ■ Why? ■ The Word ◆ know when optimality is abandoned ■ The Founder ◆ solve ‘impossible’ problems This Class ◆ sets you apart (eg, Amazon.com) Complexity ■ CPUs aren’t getting faster ■ Devices are getting smaller ■ Software is the differentiator ■ ‘Software is eating the world’ — Marc Andreessen, 2011 ■ Everything is computation Wheeler Ruml (UNH) Class 1, CS 758 – 6 / 25 The Word: Ab¯u‘Abdall¯ah Muh.ammad ibn M¯us¯aal-Khw¯arizm¯ı ■ COVID 780-850 AD Algorithms Born in Uzbekistan, ■ Algorithms Today worked in Baghdad. ■ Definition ■ Why? Solution of linear and ■ The Word ■ The Founder quadratic equations. This Class Founder of algebra. Complexity Popularized arabic numerals, decimal positional numbers → algorism (manipulating digits) → algorithm. The Compendious Book on Calculation by Completion and Balancing, 830. Wheeler Ruml (UNH) Class 1, CS 758 – 7 / 25 The Founder: Donald E. Knuth ■ COVID invented algorithm analysis Algorithms The Art of Computer ■ Algorithms Today Programming, vol. 1, 1968 ■ Definition ■ Why? ■ The Word ■ The Founder developed TEX, This Class literate programming Complexity many famous students published in MAD magazine Wheeler Ruml (UNH) Class 1, CS 758 – 8 / 25 ■ COVID Algorithms This Class ■ Relations ■ Topics ■ Course Mechanics Complexity This Class Wheeler Ruml (UNH) Class 1, CS 758 – 9 / 25 Relations ■ ■ COVID requires 531/659 (formal thinking), 515 (data structures, C) Algorithms ■ some intentional overlap! This Class ■ central (required for BS CS and BS DS) ■ Relations ■ ■ Topics same content, assignments, grading both semesters ■ Course Mechanics ■ continuous improvement! Complexity Wheeler Ruml (UNH) Class 1, CS 758 – 10 / 25 Topics ‘Greatest Hits’ ■ COVID Algorithms 1. data structures: trees, tries, hashing This Class 2. algorithms: divide-and-conquer, dynamic programming, ■ Relations greedy, graphs ■ Topics ■ Course Mechanics 3. correctness: invariants Complexity 4. complexity: time and space 5. NP-completeness: reductions Not including 1. (much) computability 2. (many) randomized algorithms 3. parallel algorithms 4. distributed algorithms 5. numerical algorithms, eg: crypto, linear algebra 6. geometric algorithms 7. on-line or ‘run forever’ algorithms 8. fancy analysis Wheeler Ruml (UNH) Class 1, CS 758 – 11 / 25 Course Mechanics ■ ■ COVID names → faces Algorithms ■ sign up sheet This Class ■ General information ■ Relations ■ Topics ◆ contact, books, C, due dates, collaboration, piazza.com ■ Course Mechanics Complexity ■ Schedule ◆ wildcard slot ■ Expectations ◆ 50/4=12.5; 50/3=16.7 ◆ 2018: median 12, mean 12.8, stddev 5.5 ■ Feedback is always needed and appreciated. ◆ eg, EOLQs. Try coming to my office hours! Wheeler Ruml (UNH) Class 1, CS 758 – 12 / 25 ■ COVID Algorithms This Class Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs Wheeler Ruml (UNH) Class 1, CS 758 – 13 / 25 Sorting ■ ■ COVID Bubble Sort Algorithms ■ Selection Sort This Class ■ Insertion Sort Complexity ■ Shell Sort ■ Sorting ■ ■ Counting Sort Merge Sort ■ Correctness ■ Heap Sort ■ Counting Sort ■ Complexity ■ Quick Sort ■ Counting Sort ■ Order Notation How to sort one million records? ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs Wheeler Ruml (UNH) Class 1, CS 758 – 14 / 25 Sorting ■ ■ COVID Bubble Sort Algorithms ■ Selection Sort This Class ■ Insertion Sort Complexity ■ Shell Sort ■ Sorting ■ ■ Counting Sort Merge Sort ■ Correctness ■ Heap Sort ■ Counting Sort ■ Complexity ■ Quick Sort ■ Counting Sort ■ Order Notation How to sort one million records? ■ O() ■ Examples ■ And Friends ■ Asymptotics How to sort one billion 16-bit integers? ■ EOLQs Wheeler Ruml (UNH) Class 1, CS 758 – 14 / 25 Sorting ■ ■ COVID Bubble Sort Algorithms ■ Selection Sort This Class ■ Insertion Sort Complexity ■ Shell Sort ■ Sorting ■ ■ Counting Sort Merge Sort ■ Correctness ■ Heap Sort ■ Counting Sort ■ Complexity ■ Quick Sort ■ Counting Sort ■ Order Notation How to sort one million records? ■ O() ■ Examples ■ And Friends ■ Asymptotics How to sort one billion 16-bit integers? ■ EOLQs How to sort one trillion 4-bit integers? Wheeler Ruml (UNH) Class 1, CS 758 – 14 / 25 Counting Sort ■ COVID For n numbers in the range 0 to k: Algorithms This Class Complexity 1. for i from 0 to k ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x ■ Counting Sort 4. increment count[x] ■ Complexity ■ Counting Sort 5. for i from 0 to k ■ Order Notation i ■ O() 6. docount[ ] times ■ Examples 7. emit i ■ And Friends ■ Asymptotics ■ EOLQs Wheeler Ruml (UNH) Class 1, CS 758 – 15 / 25 Counting Sort ■ COVID For n numbers in the range 0 to k: Algorithms This Class Complexity 1. for i from 0 to k ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x ■ Counting Sort 4. increment count[x] ■ Complexity ■ Counting Sort 5. for i from 0 to k ■ Order Notation i ■ O() 6. docount[ ] times ■ Examples 7. emit i ■ And Friends ■ Asymptotics ■ EOLQs Correctness? Complexity? Wheeler Ruml (UNH) Class 1, CS 758 – 15 / 25 Correctness ■ COVID property 1: output is in sorted order Algorithms proof sketch: output loop increments i, never decrements This Class Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Correctness ■ COVID property 1: output is in sorted order Algorithms proof sketch: output loop increments i, never decrements This Class Complexity ■ Sorting property 2: output contains same numbers as input ■ Counting Sort ■ Correctness invariant: ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Correctness ■ COVID property 1: output is in sorted order Algorithms proof sketch: output loop increments i, never decrements This Class Complexity ■ Sorting property 2: output contains same numbers as input ■ Counting Sort ■ Correctness invariant: for each value, ■ Counting Sort remaining input + sum of counts = total ■ Complexity ■ Counting Sort proof sketch: ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Correctness ■ COVID property 1: output is in sorted order Algorithms proof sketch: output loop increments i, never decrements This Class Complexity ■ Sorting property 2: output contains same numbers as input ■ Counting Sort ■ Correctness invariant: for each value, ■ Counting Sort remaining input + sum of counts = total ■ Complexity ■ Counting Sort proof sketch: ■ Order Notation initialized/established: before line 3 ■ O() ■ Examples maintained: through lines 3–4 ■ And Friends ■ Asymptotics at termination: no remaining input ■ EOLQs each number printed count times therefore, output has same numbers as input Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Counting Sort ■ COVID For n numbers in the range 0 to k: Algorithms This Class Complexity 1. for i from 0 to k ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x ■ Counting Sort 4. increment count[x] ■ Complexity ■ Counting Sort 5. for i from 0 to k ■ Order Notation i ■ O() 6. docount[ ] times ■ Examples 7. emit i ■ And Friends ■ Asymptotics ■ EOLQs Correctness? Yes. Complexity? Wheeler Ruml (UNH) Class 1, CS 758 – 17 / 25 Complexity ■ COVID RAM model: no cache Algorithms order of growth This Class worst-case Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs [ try with previous slide ] Wheeler Ruml (UNH) Class 1, CS 758 – 18 / 25 Counting Sort ■ COVID For n numbers in the range 0 to k: Algorithms This Class Complexity 1. for i from 0 to k O(k) ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x O(n) ■ Counting