Analysis of Algorithms
Total Page:16
File Type:pdf, Size:1020Kb
Analysis of Algorithms CSci 653, TTh 9:30-10:50, ISC 0248 Professor Mao, [email protected], 1-3472, McGl 114 General Information I Office Hours: TTh 11:00 – 12:00 and W 3:00 – 4:00 I Textbook: Intro to Algorithms (any edition), CLRS, McGraw Hill or MIT Press. I Prerequisites/background: Linear algebra, Data structures and algorithms, and Discrete math Use of Blackboard I Announcements I Problem sets (aka assignments or homework) I Lecture notes I Grades I Check at least weekly Lecture Basics I Lectures: Slides + board (mostly) I Lecture slides ⊂ lecture notes posted on BB I Not all taught in class are in the lecture notes/slides, e.g., some example problems. So take your own notes in class Course Organization I Mathematical foundation I Methods of analyzing algorithms I Methods of designing algorithms I Additional topics chosen from the lower bound theory, amortization, probabilistic analysis, competitive analysis, NP-completeness, and approximation algorithms Grading I About 12 problem sets: 60% I Final: 40% (in-class) Grading Policy (may be curved) I [90;100]: A or A- I [80;90): B+, B, or B- I [70;80): C+, C, or C- I [60;70): D+, D, or D- I [0;60):F Homework Submission Policy I Hardcopy (stapled and typeset in LaTex) to be submitted at the beginning of class on the due date I Extensions may be permitted for illness, family emergency, and travel to interviews/conferences. Requests must be made prior to the due date Homework Policy I Homework must be typeset in LaTex. Nothing should be handwritten, including diagrams and tables I Empty-hand policy when you discuss homework problems with your classmates I List your collaborators for every homework problem I Cite all references that help you solve a problem I In no case you should copy verbatim from other people’s work without proper attribution, as this is considered plagiarism Honor Code I ”As a member of the William and Mary community, I pledge on my honor not to lie, or steal, either in my academic or personal life. I understand that such acts violates the Honor Code and undermine the community of trust, of which we are all stewards.” I Academic honesty, the cornerstone of teaching and learning, lays the foundation for lifelong integrity. The Honor Code is, as always, in effect in this course. Please go to the ”Honor System” page in the wm.edu site to read the complete honor code document. Common functions I Monotonicity: Definitions of monotonically increasing/decreasing or strictly increasing/decreasing functions. Important note: In this course, since functions are used to represent time complexity, we restrict our attention to only increasing functions that map positive number(s) to positive number. I Ceilings and floors: dxe and bxc, where x can be any real number. x − 1 < bxc ≤ x ≤ dxe < x + 1, dn=2e + bn=2c = n. I Modular arithmetic: a mod n = a − ba=ncn. a ≡ b mod n iff a mod n = b mod n. d i I Polynomials: p(n) = ∑i=0 ai n . (Note: Coefficients ai and degree d are constants.) 0 −1 1 m n m+n I Exponentials: a = 1, a = a , a · a = a , am=an = am−n. x x2 x3 ¥ xi e = 1 + x + 2! + 3! + ··· = ∑i=0 i! . (Note: e = 2:71828:::.) x n x limn!¥ (1 + n ) = e . I Logarithms: logn = log2 n or logc n for some c we don’t care about. a log(ab) = loga + logb, log( b ) = loga − logb. log b = logc b . a logc a n n loga n logc b logc a loga b = nloga b 6= (loga b) , a = n, a = b . x2 x3 x4 ln(1 + x) = x − 2 + 3 − 4 + ···. I Factorials: n! = n · (n − 1)···2 · 1. n! = n · (n − 1)!, 0! = 1. (Recursivep definition) n n 1 Sterling’s approximation: n! = 2pn( e ) (1 + Θ( n )). (Note: Θ means having the same order of magnitude.)p n n an The following approximation also holds: n! = 2pn( e ) e , 1 1 where 12n+1 < an < 12n . logn! = Θ(nlogn). I Functional iteration: A function f applied iteratively i times to an initial argument n. Defined recursively, f (0)(n) = n and f (i)(n) = f (f (i−1)(n)) for i > 0. (Note: The distinction between f (i)(n) and f i (n).) For example, if f (n) = 2n then f (i)(n) = 2i n. ∗ (i) I The log star function: log n = minfi ≥ 0 : log n ≤ 1g, which is a very slowly growing function. log∗ 2 = 1, log∗ 4 = 2, log∗ 16 = 3, log∗ 65536 = 4, log∗ 265536 = 5. I Fibonacci numbers: F0 = 0;F1 = 1;Fi = Fi−1 + Fi−2 for i ≥ 2. p fi −^fi 1+ 5 Fi = p , where f = = 1:61803::: is called the 5 p 2 ^ 1− 5 golden ratio, f = 2 = −0:61803::: is the conjugate of f, and both are roots of equation x2 = x + 1. Asymptotic notation I Used to compare the growth rate or order of magnitude of increasing functions. “Asymptotic” describes the behavior of functions in the limit, for sufficiently large values of variables. I f (n) = O(g(n)) if 9c;n0 such that f (n) ≤ cg(n) for n ≥ n0. I f (n) = Ω(g(n)) if 9c;n0 such that f (n) ≥ cg(n) for n ≥ n0. I f (n) = Θ(g(n)) if 9c1;c2;n0 such that c1g(n) ≤ f (n) ≤ c2g(n) for n ≥ n0. I f (n) = o(g(n)) if 8c 9n0 such that f (n) < cg(n) for n ≥ n0. I f (n) = w(g(n)) if 8c 9n0 such that f (n) > cg(n) for n ≥ n0. I Remarks: I In CLRS, the above notation is defined as sets of functions. For example, f (n) 2 O(g(n)). I Comparison of growth rates of two functions: O(≤), Ω(≥), Θ(=), o(<), w(>). I f (n) = O(g(n)) iff g(n) = Ω(f (n)), and f (n) = o(g(n)) iff g(n) = w(f (n)). I f (n) = Θ(g(n)) iff f (n) = O(g(n)) and f (n) = Ω(g(n)). I f (n) = O(g(n)) if f (n) = o(g(n)), and f (n) = Ω(g(n)) if f (n) = w(g(n)). I An alternative definition for f (n) = o(g(n)) is f (n) limn!¥ g(n) = 0. Likewise, an alternative definition for f (n) f (n) = w(g(n)) is limn!¥ g(n) = ¥. I More remarks: I Asymptotic notation ignores constant factors and lower-order terms. I Rule of thumb: constant ≤ polylogarithmic ≤ polynomial ≤ exponential ≤ superexponential. p p Example: 1, plogn, lnn, (logn)2, n, nlogn, n, nlogn, n2, nloglogn, 2n, n2n, n!, 22n . Summations/Series n n n I Property of linearity: ∑i=1(cai + bi ) = c ∑i=1 ai + ∑i=1 bi and n n ∑i=1 Θ(f (i)) = Θ(∑i=1 f (k)). n 1 I Arithmetic sum/series: ∑i=1 i = 1 + 2 + ··· + n = 2 n(n + 1). I Geometric sum/series: n i 2 n r n+1−1 ∑i=0 r = 1 + r + r + ··· + r = r−1 for r 6= 1. 2 1 1 + r + r + ··· = 1−r for jrj < 1. 1 1 1 e I Harmonic series: Hn = 1 + 2 + 3 + ··· + n = lnn + g + 2n for g = 0:5772156649::: (Euler’s constant) and 0 < e < 1. Example: Prove that ln(n + 1) < Hn < lnn + 1. (Approximation by integrals) n n n n n I Binomial series: 0 + 1 + 2 + ··· + n = 2 . I Other useful sums: n 2 1 ∑i=1 i = 6 n(n + 1)(2n + 1). (A direct proof starting with i 2 ∑j=1(2j − 1) = i ) n 3 n 2 ∑i=1 i = (∑i=1 i) . (Proved by induction) n+1 n n i−1 = nx −(n+1)x +1 ∑i=1 ix (x−1)2 . (Proved by using derivatives) Proof techniques I Proving by contradiction: The following three statements are logically equivalent: 1. If A then B. 2. If not B then not A. 3. If A and not B then not C, where C is a proved fact or axiom. Example: Use contradiction to prove that (a)p There are infinitely many prime numbers and (b) 2 is irrational. I Proving by induction: The following statements are mathematically equivalent: 1. P(n) for integers n ≥ c. 2. Simple integer induction: P(c) and P(n − 1) ! P(n). (What are inductive basis, inductive hypothesis, and inductive step?) 3. General integer induction: P(c) and (8i : c ≤ i ≤ n − 1)P(i) ! P(n). Example: Use induction to prove that n 3 n 2 (a) ∑i=1 i = (∑i=1 i) and (b) Every positive composite integer can be expressed as a product of prime numbers. Solving recurrences I Recurrence is an equation or inequality that defines a function in terms of the function’s values on smaller inputs. For example, T (1) = Θ(1) (boundary condition) and n T (n) = 2T ( 2 ) + Θ(n) for n ≥ 2 (recurrence) or almost n equivalently, T (1) = 1 and T (n) = 2T ( 2 ) + n for n ≥ 2. I Remark: We may neglect some technical details due to our interest in asymptotic solutions: I Relax the integer argument requirement on functions. For example, use T (n=2) instead of T (bn=2c) or T (dn=2e). I Assume boundary condition T (n) = Θ(1) for small n if not given explicitly. Asymptotically, Θ(1) is the same as any constant c no matter how large it is.