CS 758/858: Algorithms

■ COVID Prof. Wheeler Ruml Algorithms TA Sumanta Kashyapi This Class

Complexity http://www.cs.unh.edu/~ruml/cs758

4 handouts: course info, schedule, slides, asst 1

2 online handouts: programming tips, formulas

1 physical sign-up sheet/laptop (for grades, piazza)

Wheeler Ruml (UNH) Class 1, CS 758 – 1 / 25 COVID

■ COVID

Algorithms

This Class

Complexity

■ check your Wildcat Pass before coming to campus ■ if you have concerns, let me know

Wheeler Ruml (UNH) Class 1, CS 758 – 2 / 25 ■ COVID

Algorithms ■ Algorithms Today ■ Definition ■ Why? ■ The Word ■ The Founder

This Class Complexity Algorithms

Wheeler Ruml (UNH) Class 1, CS 758 – 3 / 25 Algorithms Today

■ ■ COVID web: search, caching, crypto

Algorithms ■ networking: routing, synchronization, failover ■ Algorithms Today ■ machine learning: data mining, recommendation, prediction ■ Definition ■ Why? ■ bioinformatics: alignment, matching, clustering ■ The Word ■ ■ The Founder hardware: design, simulation, verification ■ This Class business: allocation, planning, scheduling

Complexity ■ AI: robotics, games

Wheeler Ruml (UNH) Class 1, CS 758 – 4 / 25 Definition

■ COVID Algorithm Algorithms ■ precisely defined ■ Algorithms Today ■ Definition ■ mechanical steps ■ Why? ■ ■ The Word terminates ■ The Founder ■ input and related output This Class

Complexity What might we want to know about it?

Wheeler Ruml (UNH) Class 1, CS 758 – 5 / 25 Why?

■ ■ COVID Computer scientist 6= programmer Algorithms ◆ ■ Algorithms Today understand program behavior ■ Definition ◆ have confidence in results, performance ■ Why? ■ The Word ◆ know when optimality is abandoned ■ The Founder ◆ solve ‘impossible’ problems This Class ◆ sets you apart (eg, Amazon.com) Complexity ■ CPUs aren’t getting faster ■ Devices are getting smaller ■ Software is the differentiator ■ ‘Software is eating the world’ — Marc Andreessen, 2011 ■ Everything is computation

Wheeler Ruml (UNH) Class 1, CS 758 – 6 / 25 The Word: Ab¯u‘Abdall¯ah Muh.ammad ibn M¯us¯aal-Khw¯arizm¯ı

■ COVID 780-850 AD

Algorithms Born in Uzbekistan, ■ Algorithms Today worked in Baghdad. ■ Definition ■ Why? Solution of linear and ■ The Word ■ The Founder quadratic equations.

This Class Founder of algebra.

Complexity Popularized arabic numerals, decimal positional numbers → algorism (manipulating digits) → algorithm.

The Compendious Book on Calculation by Completion and Balancing, 830.

Wheeler Ruml (UNH) Class 1, CS 758 – 7 / 25 The Founder: Donald E. Knuth

■ COVID invented algorithm analysis

Algorithms The Art of Computer ■ Algorithms Today Programming, vol. 1, 1968 ■ Definition ■ Why? ■ The Word ■ The Founder developed TEX, This Class literate programming

Complexity many famous students published in MAD magazine

Wheeler Ruml (UNH) Class 1, CS 758 – 8 / 25 ■ COVID

Algorithms

This Class ■ Relations ■ Topics ■ Course Mechanics

Complexity This Class

Wheeler Ruml (UNH) Class 1, CS 758 – 9 / 25 Relations

■ ■ COVID requires 531/659 (formal thinking), 515 (data structures, C) Algorithms ■ some intentional overlap! This Class ■ central (required for BS CS and BS DS) ■ Relations ■ ■ Topics same content, assignments, grading both semesters ■ Course Mechanics ■ continuous improvement! Complexity

Wheeler Ruml (UNH) Class 1, CS 758 – 10 / 25 Topics

‘Greatest Hits’ ■ COVID

Algorithms 1. data structures: trees, tries, hashing

This Class 2. algorithms: divide-and-conquer, dynamic programming, ■ Relations greedy, graphs ■ Topics ■ Course Mechanics 3. correctness: invariants Complexity 4. complexity: time and space 5. NP-completeness: reductions Not including 1. (much) computability 2. (many) randomized algorithms 3. parallel algorithms 4. distributed algorithms 5. numerical algorithms, eg: crypto, linear algebra 6. geometric algorithms 7. on-line or ‘run forever’ algorithms 8. fancy analysis

Wheeler Ruml (UNH) Class 1, CS 758 – 11 / 25 Course Mechanics

■ ■ COVID names → faces

Algorithms ■ sign up sheet This Class ■ General information ■ Relations ■ Topics ◆ contact, books, C, due dates, collaboration, piazza.com ■ Course Mechanics

Complexity ■ Schedule ◆ wildcard slot ■ Expectations ◆ 50/4=12.5; 50/3=16.7 ◆ 2018: median 12, mean 12.8, stddev 5.5 ■ Feedback is always needed and appreciated. ◆ eg, EOLQs. Try coming to my office hours!

Wheeler Ruml (UNH) Class 1, CS 758 – 12 / 25 ■ COVID

Algorithms

This Class

Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 13 / 25 Sorting

■ ■ COVID Bubble Sort

Algorithms ■ Selection Sort This Class ■ Insertion Sort Complexity ■ Shell Sort ■ Sorting ■ ■ Counting Sort Merge Sort ■ Correctness ■ Heap Sort ■ Counting Sort ■ Complexity ■ Quick Sort ■ Counting Sort ■ Order Notation How to sort one million records? ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 14 / 25 Sorting

■ ■ COVID Bubble Sort

Algorithms ■ Selection Sort This Class ■ Insertion Sort Complexity ■ Shell Sort ■ Sorting ■ ■ Counting Sort Merge Sort ■ Correctness ■ Heap Sort ■ Counting Sort ■ Complexity ■ Quick Sort ■ Counting Sort ■ Order Notation How to sort one million records? ■ O() ■ Examples ■ And Friends ■ Asymptotics How to sort one billion 16-bit integers? ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 14 / 25 Sorting

■ ■ COVID Bubble Sort

Algorithms ■ Selection Sort This Class ■ Insertion Sort Complexity ■ Shell Sort ■ Sorting ■ ■ Counting Sort Merge Sort ■ Correctness ■ Heap Sort ■ Counting Sort ■ Complexity ■ Quick Sort ■ Counting Sort ■ Order Notation How to sort one million records? ■ O() ■ Examples ■ And Friends ■ Asymptotics How to sort one billion 16-bit integers? ■ EOLQs How to sort one trillion 4-bit integers?

Wheeler Ruml (UNH) Class 1, CS 758 – 14 / 25 Counting Sort

■ COVID For n numbers in the range 0 to k:

Algorithms

This Class

Complexity 1. for i from 0 to k ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x ■ Counting Sort 4. increment count[x] ■ Complexity ■ Counting Sort 5. for i from 0 to k ■ Order Notation i ■ O() 6. docount[ ] times ■ Examples 7. emit i ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 15 / 25 Counting Sort

■ COVID For n numbers in the range 0 to k:

Algorithms

This Class

Complexity 1. for i from 0 to k ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x ■ Counting Sort 4. increment count[x] ■ Complexity ■ Counting Sort 5. for i from 0 to k ■ Order Notation i ■ O() 6. docount[ ] times ■ Examples 7. emit i ■ And Friends ■ Asymptotics ■ EOLQs Correctness?

Complexity?

Wheeler Ruml (UNH) Class 1, CS 758 – 15 / 25 Correctness

■ COVID property 1: output is in sorted order

Algorithms proof sketch: output loop increments i, never decrements

This Class

Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Correctness

■ COVID property 1: output is in sorted order

Algorithms proof sketch: output loop increments i, never decrements

This Class

Complexity ■ Sorting property 2: output contains same numbers as input ■ Counting Sort ■ Correctness invariant: ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Correctness

■ COVID property 1: output is in sorted order

Algorithms proof sketch: output loop increments i, never decrements

This Class

Complexity ■ Sorting property 2: output contains same numbers as input ■ Counting Sort ■ Correctness invariant: for each value, ■ Counting Sort remaining input + sum of counts = total ■ Complexity ■ Counting Sort proof sketch: ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Correctness

■ COVID property 1: output is in sorted order

Algorithms proof sketch: output loop increments i, never decrements

This Class

Complexity ■ Sorting property 2: output contains same numbers as input ■ Counting Sort ■ Correctness invariant: for each value, ■ Counting Sort remaining input + sum of counts = total ■ Complexity ■ Counting Sort proof sketch: ■ Order Notation initialized/established: before line 3 ■ O() ■ Examples maintained: through lines 3–4 ■ And Friends ■ Asymptotics at termination: no remaining input ■ EOLQs each number printed count times therefore, output has same numbers as input

Wheeler Ruml (UNH) Class 1, CS 758 – 16 / 25 Counting Sort

■ COVID For n numbers in the range 0 to k:

Algorithms

This Class

Complexity 1. for i from 0 to k ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x ■ Counting Sort 4. increment count[x] ■ Complexity ■ Counting Sort 5. for i from 0 to k ■ Order Notation i ■ O() 6. docount[ ] times ■ Examples 7. emit i ■ And Friends ■ Asymptotics ■ EOLQs Correctness? Yes.

Complexity?

Wheeler Ruml (UNH) Class 1, CS 758 – 17 / 25 Complexity

■ COVID RAM model: no cache

Algorithms order of growth This Class worst-case Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

[ try with previous slide ]

Wheeler Ruml (UNH) Class 1, CS 758 – 18 / 25 Counting Sort

■ COVID For n numbers in the range 0 to k:

Algorithms

This Class

Complexity 1. for i from 0 to k O(k) ■ Sorting 2. count[i] ← 0 ■ Counting Sort ■ Correctness 3. for each input number x O(n) ■ Counting Sort 4. increment count[x] ■ Complexity ■ Counting Sort 5. for i from 0 to k O(k + n) ■ Order Notation i ■ O() 6. docount[ ] times ■ Examples 7. emit i ■ And Friends ■ Asymptotics ■ EOLQs O(k + n + k + n) = O(2n +2k) = O(n + k) 6= O(n lg n)

Wheeler Ruml (UNH) Class 1, CS 758 – 19 / 25 Order Notation

■ COVID ignore constant factors

Algorithms ingore ‘start-up costs’ This Class upper bound Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 20 / 25 Order Notation

■ COVID ignore constant factors f(n) = O(g(n))

Algorithms ingore ‘start-up costs’ c·g(n) This Class upper bound Complexity ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity f(n) ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics n0 ■ EOLQs eg, running time is O(n log n)

Wheeler Ruml (UNH) Class 1, CS 758 – 20 / 25 O()

■ COVID

Algorithms O(g(n)) = {f(n): there exist positive constants c, n0 This Class such that f(n) ≤ cg(n) for all n ≥ no} Complexity ■ Sorting ■ Counting Sort We can upper-bound (the tail of) f by scaling up g. ■ Correctness ■ Counting Sort ■ Complexity Note non-transitive use of =. Pronounced ‘is’. ■ Counting Sort ■ Order Notation ■ O() Eg: ■ Examples 2 80 ■ And Friends 1. 0.002x − 35, 456x +2 ■ Asymptotics 2 3 ■ EOLQs 2. O(n ) vs O(n ) 3. O(2n) vs O(3n) 4. O(2n) vs O(2n+2) vs O(22n) vs O(nn) “What is n?”

Wheeler Ruml (UNH) Class 1, CS 758 – 21 / 25 Examples

3 2 ■ COVID is n = O(n ) 2 20 Algorithms 0.2x − 456x +2 This Class 10n2 +5n Complexity O(n2) vs O(n3) ■ Sorting ■ Counting Sort ■ Correctness ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 22 / 25 And Friends

■ COVID Upper bound (‘order of’): Algorithms O(g(n)) = {f(n): there exist positive constants c, n0 This Class such that f(n) ≤ cg(n) for all n ≥ no} Complexity ■ Sorting ■ Counting Sort Lower bound: ■ Correctness Ω(g(n)) = {f(n): there exist positive constants c, n0 ■ Counting Sort cg n f n n n ■ Complexity such that ( ) ≤ ( ) for all ≥ o} ■ Counting Sort ■ Order Notation ■ O() Tight bound: ■ Examples Θ(g(n)) = {f(n): there exist positive constants c1,c2, n0 ■ And Friends ■ Asymptotics such that c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ no} ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 23 / 25 Asymptotics

■ COVID Upper bound (’dominated by’): f(n) Algorithms →∞ o(g(n)) = {f(n) : limn g(n) =0} This Class

Complexity ■ Sorting Lower bound (’dominates’): ■ Counting Sort f(n) ω(g(n)) = {f(n) : limn→∞ = ∞} ■ Correctness g(n) ■ Counting Sort ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 24 / 25 EOLQs

■ ■ COVID What’s still confusing?

Algorithms ■ What question didn’t you get to ask today? This Class ■ What would you like to hear more about? Complexity ■ Sorting Please write down your most pressing question about algorithms ■ Counting Sort ■ Correctness and put it in the box on your way out. ■ Counting Sort Thanks! ■ Complexity ■ Counting Sort ■ Order Notation ■ O() ■ Examples ■ And Friends ■ Asymptotics ■ EOLQs

Wheeler Ruml (UNH) Class 1, CS 758 – 25 / 25