CS210-Data Structures-Module-21-Quick-Sort

CS210-Data Structures-Module-21-Quick-Sort

Data Structures and Algorithms (CS210A) Lecture 21 • Analyzing average running time of Quick Sort 1 Overview of this lecture Main Objective: • Analyzing average time complexity of QuickSort using recurrence. – Using mathematical induction. – Solving the recurrence exactly. • The outcome of this analysis will be quite surprising! Extra benefits: • You will learn a standard way of using mathematical induction to bound time complexity of an algorithm. You must try to internalize it. 2 QuickSort 3 Pseudocode for QuickSort(푺) QuickSort(푺) { If (|푺|>1) Pick and remove an element 풙 from 푺; (푺<풙, 푺>풙) Partition(푺,풙); return( Concatenate(QuickSort(푺<풙), 풙, QuickSort(푺>풙)) } 4 Pseudocode for QuickSort(푺) When the input 푺 is stored in an array QuickSort(푨,풍, 풓) { If (풍 < 풓) 풊 Partition(푨,풍,풓); QuickSort(푨,풍, 풊 − ퟏ); QuickSort(푨,풊 + ퟏ, 풓) } Partition : 풙푨[풍] as a pivot element, permutes the subarray 푨[풍 … 풓] such that elements preceding 풙 are smaller than 풙, 푨[풊]= 풙, and elements succeeding 풙 are greater than 풙. 5 Analyzing average time complexity of QuickSort Part 1 Deriving the recurrence 6 Analyzing average time complexity of QuickSort Assumption (just for a neat analysis): • All elements are distinct. • Each recursive call selects the first element of the subarray as the pivot element. 7 Analyzing average time complexity of QuickSort A useful Fact: Quick sort is a comparison based algorithm. 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 6 11 42 37 24 5 16 27 2 15 20 49 41 29 4 23 36 3 푒 3 푒4 푒9 푒8 푒6 푒2 푒5 푒7 푒1 푒3 푒4 푒9 푒8 푒6 푒2 푒5 푒7 푒1 Let 푒푖 : 푖th smallest element of 푨. Observation: The execution of Quick sort depends upon the permutation of 푒푖’s and not on the values taken by 푒푖’s. 8 Analyzing average time complexity of QuickSort 푻(풏) : Average running time for Quick sort on input of size 풏. (average over all possible permutations of {푒1, 푒2,… ,푒푛}) 1 Hence, 푻(풏)= 푸(휋), 풏! 휋 where 푸(휋) is the time complexity (or no. of comparisons) when the input is permutation 휋. Calculating 푻(풏) from definition/scratch is impractical, if not impossible. 9 Analyzing average time complexity of QuickSort Permutations beginning with 푒2 Permutations beginning with 푒1 Permutations beginning with 푒3 All 풏! permutations of {푒1, 푒2,… ,푒푛} Let P(풊) be the set of all those permutations of {푒1, 푒2,… ,푒푛} that begin with 푒푖. Question: What fraction of all permutations constitutes P(풊) ? 1 Answer: 풏 Let 푮(풏, 풊) be the average running time of QuickSort over P(풊). Question: What is the relation between 푻(풏) and 푮(풏, 풊)’s ? ퟏ 푛 Answer: 푻(풏) = 풏 푖<1 푮(풏, 풊) Observation: We now need to derive an expression for 푮(풏, 풊).For this purpose, we need to have a closer look at the execution of QuickSort over P(풊). 10 Quick Sort on a permutation from P(풊). > 푒 0 1 2 3 4 5 6 7 8 푖 푨 < 푒푖 푒푖 What happens during Partition(푨,ퟎ,ퟖ) 11 Quick Sort on a permutation from P(풊). (풏 − ퟏ)! 0 1 2 3 4 5 6 7 8 P (풊) 푨 푒 Many-to-one 푖 Is it also a mapping uniform mapping ? Reasons: 0 1 2 3 4 5 6 7 8 - Partition() isYes “well -defined” S(풊) - Partition() just compares pivot with other elements. (풊 − ퟏ)!⨯ (풏 − 풊)! 푒푖 푒 … 푒 푒1… 푒푖;1 푖:1 푛 Lemma 1: 풏 − ퟏ S(풊): Permutations resulting from Partition(). There are exactly ( ) permutations from P(풊) that get mapped to one permutation in S(풊). 풊 − ퟏ 12 Homework: Try your best to prove Lemma 1. If you have still any doubt, please meet me. Quick Sort on a permutation from P(풊). (풏 − ퟏ)! 0 1 2 3 4 5 6 7 8 P (풊) 푨 Many-to-one 푒푖 Using Lemma 1 Can mapping you now express 푮(풏, 풊) recursively ? 0 1 2 3 4 5 6 7 8 S(풊) (풊 − ퟏ)!⨯ (풏 − 풊)! 푒푖 푒 … 푒 푒1… 푒푖;1 푖:1 푛 Lemma 1: 풏 − ퟏ There are exactly ( ) permutations from P(풊) that get mapped to one permutation in S(풊). 풊 − ퟏ 13 Analyzing average time complexity of QuickSort 푮(풏, 풊) = 푻(풊 − ퟏ) + 푻(풏 − 풊) + d 풏 ----1 We showed previously that : ퟏ 푛 푻(풏) = 풏 푖<1 푮(풏, 풊) ----2 Question: Can you express 푻(풏) recursively using 1 and 2? ퟏ 푛 푻(풏) = 풏 푖<1(푻(풊 − 1) + 푻(풏 − 풊)) + d풏 푻(ퟏ) = c 14 Analyzing average time complexity of QuickSort Part 2 Solving the recurrence through mathematical induction 15 푻(ퟏ) = c ퟏ 푛 푻(풏) = 풏 푖<1(푻(풊 − 1) + 푻(풏 − 풊)) + d풏 ퟐ 푛;1 = 풏 푖<1 푻(풊) + d풏 Assertion A(풎): 푻(풎) ≤ a풎 퐥퐨퐠 풎 + b for all 풎 ≥ 1 Base case A(ퟎ) : Holds for b ≥ c Induction step: Assuming A(풎) holds for all 풎< 풏, we have to prove A(풏). ퟐ 푛;1 푻(풏) ≤ 풏 푖<1 (a풊 퐥퐨퐠 풊 + b) + d풏 ퟐ 푛;1 ≤ 풏( 푖<1 a풊 퐥퐨퐠 풊 ) + 2b + d풏 ퟐ 푛/2 푛;1 = ( a풊 퐥퐨퐠 풊 + 푛 a풊 퐥퐨퐠 풊 ) + 2b + d풏 풏 푖<1 푖< :1 2 ퟐ 푛/2 푛;1 ≤ ( a풊 퐥퐨퐠 풏/ퟐ + 푛 a풊 퐥퐨퐠 풏 ) + 2b + d풏 풏 푖<1 푖< :1 2 ퟐ 푛;1 푛/2 = 풏( 푖<1 a풊 퐥퐨퐠 풏 − 푖<1 a풊) + 2b + d풏 풏 풏 ( :ퟏ) ퟐ 풏(풏;ퟏ) ퟐ ퟐ = ( a 퐥퐨퐠 풏 − a) + 2b + d풏 풏 ퟐ ퟐ 풏 ≤ a 풏 − ퟏ 퐥퐨퐠 풏 − a + 2b + d풏 ퟒ 풏 = a풏퐥퐨퐠 풏+ b − a + b + d풏 ퟒ ≤ a풏퐥퐨퐠 풏+ b for a >ퟒ(b + d) 16 Analyzing average time complexity of QuickSort Part 3 Solving the recurrence exactly 17 Some elementary tools 푛 ퟏ 퐇 풏 = 풊 푖<1 Question: How to approximate 퐇(풏) ? Answer: 퐇(풏) 퐥퐨퐠풆 풏 + ᵞ, as 풏 increases where ᵞ is Euler’s constant ~0.58 Look at this figure, and relate Hint: 1 it to the curve for function f(x)= 1/x and its integration… 1/2 1/3 1/4 1/5 1/6 We shall calculate average number of comparisons during QuickSort using: • our knowledge of solving recurrences by substitution • our knowledge of solving recurrence by unfolding • our knowledge of simplifying a partial fraction (from JEE days) Students should try to internalize the way the above tools are used. 18 푻(풏) : average number of comparisons during QuickSort on 풏 elements. 푻(ퟏ) = ퟎ, 푻(ퟎ) = ퟎ, ퟏ 푛 푻(풏) = 풏 푖<1(푻(풊 − 1) + 푻(풏 − 풊)) + 풏 − ퟏ ퟐ 푛 = 풏 푖<1(푻(풊 − 1)) + 풏 − ퟏ 푛 풏푻(풏) = 2 푖<1(푻(풊 − 1)) + 풏(풏 − ퟏ) -----1 Question: How will this equation appear for 풏 − ퟏ ? 푛;1 (풏 − ퟏ)푻(풏 − ퟏ) = 2 푖<1 (푻(풊 − 1)) + (풏 − ퟏ)(풏 − ퟐ) -----2 Subtracting 2 from 1, we get 풏푻(풏) − (풏 − ퟏ)푻(풏 − ퟏ) = 2 푻(풏 − ퟏ) + 2(풏 − ퟏ) 풏푻(풏) − (풏 + ퟏ)푻(풏 − ퟏ) = 2(풏 − ퟏ) Question: How to solve/simplify it further ? 푻(풏) 푻(풏;1) ퟐ(풏−1) − = 풏+1 풏 풏(풏+1) 19 푻(풏) 푻(풏−1) ퟐ(풏−1) − = 풏:1 풏 풏(풏+1) ퟐ(풏;1) 푻(풎) 품(풏) − 품 풏 − ퟏ = where 품 풎 = 풏(풏:1) , 풎:1 Question: How to simplify RHS ? ퟐ(풏;1) ퟐ 풏:1 ;4 = = 풏(풏:1) 풏(풏:1) ퟐ 4 = − 풏 풏(풏:1) ퟐ 4 4 = − + 풏 풏 풏:1 4 2 = − 풏:1 풏 4 2 품(풏) − 품 풏 − ퟏ = − 풏:1 풏 20 4 2 품(풏) − 품 풏 − ퟏ = − 풏:1 풏 Question: How to calculate 품(풏) ? 4 2 품(풏 − ퟏ) − 품 풏 − ퟐ = − 풏 풏;ퟏ 4 2 품(풏 − ퟐ) − 품 풏 − ퟑ = − 풏;ퟏ 풏;ퟐ … = … 4 2 품(ퟐ) − 품 ퟏ = − 3 ퟐ 4 2 품(ퟏ) − 품 ퟎ = − 2 1 4 ퟏ 4 ퟏ Hence 품(풏) = + (2 풏 ) − 2 = + (2 풏 ) − 4 풏:1 풋<ퟐ 풋 풏:1 풋<ퟏ 풋 4 = + 2퐇(풏) − 4 풏:1 4 푻(풏) = (풏 + 1) ( + 2퐇(풏) − 4) 풏:1 = 2(풏 + ퟏ)퐇(풏) − 4풏 21 푻(풏) = 2(풏 + ퟏ)퐇(풏) − 4풏 = 2(풏 + 1) 퐥퐨퐠풆 풏 + 1.16 (풏 + ퟏ) − ퟒ풏 = 2풏 퐥퐨퐠풆 풏 − 2.84 풏 + O(1) = 2풏 퐥퐨퐠풆 풏 Theorem: The average number of comparisons during QuickSort on 풏 elements approaches 2풏 퐥퐨퐠풆 풏 − 2.84 풏. = 1.39 풏 퐥퐨퐠ퟐ 풏 − O(풏) The best case number of comparisons during QuickSort on 풏 elements = 풏 퐥퐨퐠ퟐ 풏 The worst case no. of comparisons during QuickSort on 풏 elements = 풏(풏 − ퟏ) 22 Quick sort versus Merge Sort No. of Comparisons Merge Sort Quick Sort Average case 풏 퐥퐨퐠 풏 ퟐ 1.39 풏 퐥퐨퐠ퟐ 풏 Best case 풏 퐥퐨퐠 풏 ퟐ 풏 퐥퐨퐠ퟐ 풏 Worst case 풏 퐥퐨퐠ퟐ 풏 풏(풏 − ퟏ) After seeing this table, no one would prefer Quick sort to Merge sort But Quick sort is still the most preferred algorithm in practice. Why ? You will find the answer yourself in the next programming assignment 23 .

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    23 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us