算法设计与分析 Lecture 2: Asymptotic Notation 卢杨 厦门大学信息学院计算机科学系 [email protected] Asymptotic Notation ¡ Intuitively, just look at the dominant term. � � = 0.1�! + 10�" + 5� + 25 ¡ Drop lower-order terms 10�! + 5n + 25. ¡ Ignore constant 0.1. ¡ But we can’t say that � � equals to �!. ¡ It grows like �". But it doesn’t equal to �". ¡ We define asymptotic notations (渐进符号) like � � = Θ(�!) to describe the asymptotic running time of an algorithm. ¡ “Asymptotic” here means “as something tends to infinity”, as we want to compare algorithms for very large �. 1 Logarithm Review Definition ¡ Useful identities for all real � > 0, � > 0, � > 0, and �, and where log# � is the unique number � s.t. �$ = �. logarithm bases are not 1： ¡ log (��) = log � + log � ¡ Notations: , , - / ¡ log. � = �log. � ¡ lg � = log! � (binary logarithm) ) ¡ ln� = log � (natural logarithm) ¡ log = − log � " . + . # # ¡ lg � = lg � (exponentiation) 0) ¡ log. � = log+ � ¡ lg lg � = lg(lg �) (composition) ¡ �%&'" , = �%&'" + ¡ Derivative: %&'# + ¡ log. � = $ %&' ( ) %&'# . ¡ ! = $( ( %* + ¡ � = �%&'" + 2 Big O Notation Definition 2.2 For a given complexity function �(�), � � � is the set of complexity functions � � for which there exists some positive real constant � and some nonnegative integer �! such that for all � ≥ �!, 0 ≤ � � ≤ ��(�). ¡ � � � is a set of functions in terms of � � that satisfy the definition. ¡ If � � = � � � , it represents that �(�) is an element in � � � . We say that � � is “big O (大O)” of � � . ¡ Strictly, we should use “∈” instead of “=”. However, it is conventional to use “=” for asymptotic notations. 3 Big O Notation ��(�) ¡ No matter how large � � = �(�(�)) � � is, it will eventually be smaller than �� � for some �(�) � and some �#. ¡ Big O notation describes an upper bound. We use it to bound the worst- case running time of an algorithm on arbitrary inputs. �! � 4 Image source: 图2.2, 张德富, 算法设计与分析, 国防工业出版社, 2009. Display of Growth of Functions 5 Image source: http://bigocheatsheet.com/img/big-o-complexity-chart.png Big O Notation Example 1 We show that �" + 10� = �(�"). Because, for � ≥ 1, �" + 10� ≤ �" + 10�" = 11�", we can take � = 11 and �# = 1 to obtain our result. ¡ To show a function is in big O of another function, the key is to find a specific value of � and �# that make the inequality hold. ¡ More examples of functions in �(�"): ¡ �!, �! + �, �! + 1000�, 1000�! + 1000�, �, �/1000, �%.''''', �!/ lg lg lg �. 6 Classroom Exercise Use the definition of Big O notation to show: Is 2", = �(2,)? 7 Classroom Exercise Proof: We prove it by contradiction. Assume there exist constants � > 0 and �! ≥ 0, such that 2"# ≤ �2#, for all � ≥ �!. Then 2"# = 2#2# ≤ �2#, 2# ≤ �. # But we can’t find any constant � is greater than 2 for all � ≥ �!. So the assumption leads to a contradiction. Then we can certify that 2"# ≠ �(2#). How about 2#$% = �(2#)? 8 Big Ω Notation Definition 2.3 For a given complexity function �(�), Ω � � is the set of complexity functions � � for which there exists some positive real constant � and some nonnegative integer �# such that for all � ≥ �#, 0 ≤ ��(�) ≤ � � . ¡ Ω � � is the opposite of � � � . ¡ If � � = Ω � � , it represents that �(�) is an element in Ω � � . We say that � � is “big Ω (大Ω)” of � � . 9 Big Ω Notation ¡ No matter how small � � = Ω(�(�)) � � is, it will eventually be larger �(�) than �� � for some � and some �#. ¡ Big Ω notation ��(�) describes an lower bound. We use it to bound the best-case running time of an algorithm on arbitrary inputs. �! � 10 Image source: 图2.3, 张德富, 算法设计与分析, 国防工业出版社, 2009. Big Θ Notation Definition 2.1 For a given complexity function �(�), Θ � � is the set of complexity functions � � for which there exists some positive real constants �. and �" and some nonnegative integer �# such that, for all � ≥ �#, 0 ≤ �.� � ≤ � � ≤ �"�(�). ¡ If � � = Θ � � , we say that � � is “big Θ (大Θ)” or has the same order (数量级) of � � . ¡ Θ � � = � � � ∩ Ω � � . 11 Relation between Big O, Big Ω and Big Θ ¡ Now we have O, Θ, and Ω. Intuitively, they just like “≤”, “=”, and “≥” for complexity functions. 12 Image source: Figure 1.6, Richard E. Neapolitan, Foundations of Algorithms (5th Edition), Jones & Bartlett Learning, 2014 Big Θ Notation � � = Θ(�(�)) ¡ � � = Θ � � implies � �(�) � � = � � � and # � � = Ω � � . ¡ Big Θ can also be used to �(�) bound the worst-case time complexity. ¡ For insertion sort, the worst- �"�(�) case is both Θ �! and � �! . ¡ However, we usually use Big O notation because we don’t care the best-case. �! � 13 Image source: 图2.1, 张德富, 算法设计与分析, 国防工业出版社, 2009. Properties of Asymptotic Notations Theorem 2.1 For any two functions �(�) and �(�), �(�) = Θ � � if and only if �(�) = � � � and �(�) = Ω � � . ¡ Θ = O and Ω. Theorem 2.2 For any two functions �.(�) and �"(�), if �. � = � �. � and �" � = � �" � , we have �. � + �" � = �(max {�. � , �" � }). ¡ Pick the larger one. 14 Properties of Asymptotic Notations ¡ Transitivity (传递性) ¡ Reflexivity (自反性) ¡ If � � = Θ � � and � � = ¡ If � � = Θ � � . Θ ℎ � then � � = Θ ℎ � . ¡ Same for O and Ω. ¡ Ω Same for O and . ¡ Symmetry (对称性) ¡ Additivity (可加性) ¡ � � = Θ � � if and only if ¡ If � � = Θ ℎ � and � � = � � = Θ � � . Θ ℎ � then � � + �(�) = ¡ Not hold for O and Ω. Θ ℎ � . ¡ Same for O and Ω. 15 Properties of Asymptotic Notations ¡ Consider the following ordering of complexity categories: Θ lg � Θ � Θ �lg � Θ �" Θ �& Θ �' Θ �# Θ �# Θ(�!) where � ≥ � ≥ 2 and � ≥ � ≥ 1. ¡ If � � is to the left of � � in the above sequence, then � � = � � � ¡ Notice: Big Θ is a set of functions. We can’t say Θ lg � < Θ � . 16 Properties of Asymptotic Notations Example 2 % Given � � = �(� − 1), prove that � � = Θ(�") " Proof: By the property, we first show that � � = �(�"): % % % % % � � − 1 = �" − � ≤ �" (for � = and � = 0). " " " " " ! Then we show that � � = Ω �" : % % % % % % % % � � − 1 = �" − � ≥ �" − � � = �" (for � = and � = 2). " " " " " " ( ( ! Thus � � = Θ(�"). 17 Using Limit to Determine Order ¡ In addition to proving by definition, we can also use limit to get asymptotic notations. = � implies � � = Θ � � if 0 < � < ∞ �(�) lim ≠ ∞ implies � � = � � � ,→1 �(�) ≠ 0 implies � � = Ω � � 18 Using a Limit to Determine Order Example 3 . Compare the orders of growth of �(� − 1) and �". " 1 �(� − 1) 1 �" − � 1 1 1 lim 2 = lim = lim (1 − ) = , ,→1 �" 2 ,→1 �" 2 ,→1 � 2 . Thus, � � − 1 = Θ(�"). " 19 Classroom Exercise Compare the orders of growth of �, and �,, when � > � > 0 20 Classroom Exercise Solution: �, � , lim = lim = 0. ,→1 �, ,→1 � 2 The limit is 0 because 0 < < 1. Thus, �, = � �, . 3 21 Using a Limit to Determine Order 4(,) ¡ When calculating lim , how to deal with the following ,→1 5(,) cases? lim �(�) = lim �(�) = 0 or ± ∞ ,→1 ,→1 22 Using a Limit to Determine Order 23 Image source: https://tieba.baidu.com/p/5933589166 Using a Limit to Determine Order L‘Hôpital’s Rule (洛必达法则) If �(�) and �(�) are both differentiable with derivatives �′(�) and �′(�), respectively, and if lim � � = lim � � = 0 or ± ∞, 6→1 6→1 then �(�) �′(�) lim = lim , 6→1 �(�) 6→1 �′(�) whenever the limit on the right exists. 24 Using a Limit to Determine Order Example 4 � log � 1 $ = lg � = �(�) �� � ln � because lg � �(lg �)/�� 1/(� ln2) lim = lim = lim = 0. 6→1 � 6→1 ��/�� 6→1 1 25 Exercises Show the correctness of the following statements. ¡ lg � = � � ¡ � = �(� lg �) ¡ � lg � = �(�") ¡ 2, = Ω(5?@ ,) ¡ lg! � = �(�#.B) 26 Conclusion After this lecture, you should know: ¡ Why do we need asymptotic notation? ¡ What are the meaning of these asymptotic notations big O, big Θ, or big Ω? ¡ How to prove a complexity function is big O, big Θ, or big Ω? ¡ How to compare the order of two complexity function? 27 Homework ¡ Page 19 2.1 2.2 2.3 2.9 28 谢谢 有问题欢迎随时跟我讨论 29.